Hi
Thanks for the answer.
I have developed a log file analyzer for RTPIS (Real Time Passenger
Information System) system, where buses drive lines and the system try
to estimate the arrival times to the bus stops. There are many different
log files (and events) and analyzing situation can be very complex. Also
spatial data can be included to the log data.
The analyzer also has a query (or analyzing) language, which describes a
expected behavior. This can be a requirement of system. Analyzer can be
think to be also a test oracle.
I have published a paper (SPLST'15 conference) about my analyzer and its
language. The paper is maybe too technical, but it is found:
http://ceur-ws.org/Vol-1525/paper-19.pdf
I do not know yet where it belongs. I think it can be some "CEP with
delays". Or do you know better ?
My analyzer can also do little bit more complex and time-consuming
analyzings? There are no a need for real time.
And it is possible to do "CEP with delays" reasonably some existing
analyzer (for example Spark) ?
Regards
PhD student at Tampere University of Technology, Finland, www.tut.fi
<http://www.tut.fi/>
Esa Heikkinen
27.4.2016, 15:51, Michael Segel kirjoitti:
Spark and CEP? It depends…
Ok, I know that’s not the answer you want to hear, but its a bit more
complicated…
If you consider Spark Streaming, you have some issues.
Spark Streaming isn’t a Real Time solution because it is a micro batch
solution. The smallest Window is 500ms. This means that if your
compute time is >> 500ms and/or your event flow is >> 500ms this
could work.
(e.g. 'real time' image processing on a system that is capturing 60FPS
because the processing time is >> 500ms. )
So Spark Streaming wouldn’t be the best solution….
However, you can combine spark with other technologies like Storm,
Akka, etc .. where you have continuous streaming.
So you could instantiate a spark context per worker in storm…
I think if there are no class collisions between Akka and Spark, you
could use Akka, which may have a better potential for communication
between workers.
So here you can handle CEP events.
HTH
On Apr 27, 2016, at 7:03 AM, Mich Talebzadeh
<mich.talebza...@gmail.com <mailto:mich.talebza...@gmail.com>> wrote:
please see my other reply
Dr Mich Talebzadeh
LinkedIn
/https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
http://talebzadehmich.wordpress.com
<http://talebzadehmich.wordpress.com/>
On 27 April 2016 at 10:40, Esa Heikkinen
<esa.heikki...@student.tut.fi <mailto:esa.heikki...@student.tut.fi>>
wrote:
Hi
I have followed with interest the discussion about CEP and Spark.
It is quite close to my research, which is a complex analyzing
for log files and "history" data (not actually for real time
streams).
I have few questions:
1) Is CEP only for (real time) stream data and not for "history"
data?
2) Is it possible to search "backward" (upstream) by CEP with
given time window? If a start time of the time window is earlier
than the current stream time.
3) Do you know any good tools or softwares for "CEP's" using for
log data ?
4) Do you know any good (scientific) papers i should read about CEP ?
Regards
PhD student at Tampere University of Technology, Finland,
www.tut.fi <http://www.tut.fi/>
Esa Heikkinen
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
<mailto:user-unsubscr...@spark.apache.org>
For additional commands, e-mail: user-h...@spark.apache.org
<mailto:user-h...@spark.apache.org>
The opinions expressed here are mine, while they may reflect a
cognitive thought, that is purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com <http://hotmail.com>
