Greetings, all,
Sorry for the late reply.
First of all, I want to thank Dominik for his help and all of you for your
effort in creating better wrapper for Siddhi.
In general, I really like the changes that were made - It is easier to create
StreamPipes element that utilizes Siddhi and to write queries.
Also, queries with simple lists now work properly.
However, I stumbled upon few problems:
* In the StreamPipes Editor, when I connect element with Siddhi query to
the rest of the pipeline requiredPropertyWithUnaryMapping is not rendered
properly. It shows that element's mapping Label name, instead of available
Event properties of element to which it was connected [1].
[cid:17983475-1bd9-4d29-a3e7-a98c677681ca] [1]
*
Function makeEventType of EventTypeGenerator gets called for every property of
an Event that gets received in element with Siddhi query, regardless of whether
it was required or not. I do not think that should be the case (this may be the
case for every other element as well, but I have not checked). Consequently,
this function gets called for complex list that is not even required by this
element and it throws an exception, because it cannot work with complex lists.
(I know that in onInvocation function of ElementController you can remove these
'unwanted' properties, but how can I achieve that with current implementation
of Siddhi wrapper?)
* The implementation of simple lists in Siddhi wrapper works well. But
there are only two implemented functions for lists (contains and collect). I
believe it would be good to implement other commonly used list functions in
Siddhi, such as size(), add(), remove(), etc. In addition, having support for
complex lists would be good, as well
* If I noticed correctly, the function count() is not supported by the
object model of Siddhi EPL in this version (e.g., "from ... select count() as
numOfOccurrences insert into ..."). It would be good to go over basic
functions/features of Siddhi EPL and make sure that they get implemented first.
In addition, Siddhi has a lot of functions/features and extensions regarding
Complex Event Processing and covering all of them in StreamPipes may be
exhausting and, generally, not desired. Is there a plan to integrate other
functions in object model (regarding lists and, possibly, other extensions) and
which ones? Or, will we ('developer' users) need to implement them by ourselves
(somehow?) in case we need them?
Let me know if you have any questions.
Kindest regards,
Branislav
________________________________
From: Dominik Riemer <[email protected]>
Sent: Monday, October 26, 2020 22:11
To: [email protected] <[email protected]>
Subject: Improvements to the Siddhi wrapper
Hi all,
Branislav has recently asked for list support in the Siddhi wrapper (e.g.,
filter a list for contained items).
I took the opportunity to extend and improve the Siddhi wrapper in some
ways:
- Support for lists (see example processors at [1, 2])
- Possibility to create a new Siddhi processor from a single class (no
controller and parameter class needed anymore, there is also a prototype of
the same structure available for the standalone wrapper
(StreamPipesDataProcessor)
- I added a simple object model to create Siddhi EPL statements instead of
the previous string-based approach. This should the creation of Siddhi
processors less error-prone. The object model is still rather incomplete and
undocumented, but it should be easy to extend in the future - the code is at
[3]
- Improved code structure of the whole wrapper to make it easier to extend
in the future
What do you think? Any feedback is welcome!
Also @Grainier, as you are the Siddhi expert, in case you have any ideas for
improvements, let me know!
Dominik
[1]
https://github.com/apache/incubator-streampipes-extensions/blob/dev/streampi
pes-processors-filters-siddhi/src/main/java/org/apache/streampipes/processor
s/siddhi/listfilter/ListFilter.java
[2]
https://github.com/apache/incubator-streampipes-extensions/blob/dev/streampi
pes-processors-filters-siddhi/src/main/java/org/apache/streampipes/processor
s/siddhi/listcollector/ListCollector.java
[3]
https://github.com/apache/incubator-streampipes/tree/dev/streampipes-wrapper
-siddhi/src/main/java/org/apache/streampipes/wrapper/siddhi/query
