Re: Issue with CEP library

2017-10-05 Thread Ajay Krishna
Hi Kostas, I noticed that you commented on FLINK-7549 and FLINK-7606. I was monitoring both these JIRAs. I was always using time characteristic as event time like you had suggested but I continue to see patterns not getting detected. Could you help shed more light on this ? I had shared some

Re: Issue with CEP library

2017-09-30 Thread Ajay Krishna
Hi Kostas, Here is an example of a simple event I am trying to detect. The first and last line are the interesting points/events. The CEP library is not able to detect something like that. 4> (96,Sat Sep 30 22:30:25 UTC 2017,complex event,Low,32.781082,-117.01864,12.0,20.0) 4> (96,Sat Sep 30

Re: Issue with CEP library

2017-09-28 Thread Kostas Kloudas
Hi Ajay, After reading all the data from your source, could you somehow tell your sources to send a watermark of Long.MaxValue (or a high value)?? I am asking this, just to see if the problem is that the data is simply buffered inside Flink because there is a problem with the timestamps and

Re: Issue with CEP library

2017-09-28 Thread Ajay Krishna
Hi Kostas, Thank you for reaching out and for the suggestions. Here are the results 1. Using an env parallelism of 1 performed similar with the additional problem that there was significant lag in the kafka topic 2. I removed the additional keyBy(0) but that did not change anything 3. I also

Re: Issue with CEP library

2017-09-28 Thread Kostas Kloudas
Hi Ajay, I will look a bit more on the issue. But in the meantime, could you run your job with parallelism of 1, to see if the results are the expected? Also could you change the pattern, for example check only for the start, to see if all keys pass through. As for the code, you apply

Issue with CEP library

2017-09-27 Thread Ajay Krishna
Hi, I've been only working with flink for the past 2 weeks on a project and am trying using the CEP library on sensor data. I am using flink version 1.3.2. Flink has a kafka source. I am using KafkaSource9. I am running Flink on a 3 node AWS cluster with 8G of RAM running Ubuntu 16.04. From the