Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Jayesh Senjaliya]

Right, this can work. Thanks for providing detailed queries...

but there is an issue for me if i remove rows from that table, that can be
used by other policies as central temporary repo.

so over the weekend i explored the code, and think this can put up the
feature to access the expired events into pattern query.

please notice there is a slight difference between using absence pattern
and accessing expired events in pattern query.

so I added this code change to use "select expired events into..." in
pattern query to give the required result here. I think that can be useful.

Issue: https://github.com/wso2/siddhi/issues/568
PR: https://github.com/wso2/siddhi/pull/569

Thanks
Jayesh




On Sat, Sep 30, 2017 at 10:43 PM, Grainier Perera  wrote:

> Hi Jay,
>
> Since you are maintaining a jobTable, you can use that to identify
> completed and non-completed jobs in a given time frame. Refer to the below
> snippet;
>
> @Plan:name('ExecutionPlan')
>
>
>> @Import('subStream:1.0.0')
>> define stream subStream (pid int, sid int, status string);
>
>
>> @Import('pubStream:1.0.0')
>> define stream pubStream (pid int, status string);
>
>
>> @Export('jobsNotCompletedInTime:1.0.0')
>> define stream jobsNotCompletedInTime (pid int, sid int);
>
>
>> @Export('jobsCompletedInTime:1.0.0')
>> define stream jobsCompletedInTime (pid int, sid int);
>
>
>> @Export('foo:1.0.0')
>> define stream foo (pid int, sid int);
>
>
>> @From(eventtable='rdbms', jdbc.url='jdbc:mysql://localhost:3306/pubsub',
>> username='root', password='root', driver.name='com.mysql.jdbc.Driver',
>> table.name='jobTable')
>> define table jobTable(pid int, sid int);
>
>
>> from pubStream as p join jobTable as t
>> on p.pid == t.pid
>> select p.pid, t.sid
>> insert into allPSJobs;
>
>
>
> -- this will emit an event after 10 sec once a publish event arrives
>
> from pubStream#window.time(10 sec)
>> select *
>> insert expired events into *expiredPublisherEvents*;
>
>
>
> -- if the pattern matches within 10 sec, that means the job completed
>> within time
>
> from every (a=allPSJobs) -> s=subStream[pid==a.pid and sid == a.sid]
>> within 10 sec
>> select s.pid, s.sid
>> insert into *jobsCompletedInTime*;
>
>
>> -- delete completed jobs from the table, then what's left in the table
>> are the uncompleted jobs
>> from jobsCompletedInTime
>> delete jobTable
>> on jobTable.pid == pid and jobTable.sid == sid;
>
>
>> -- so if we join expiredPublisherEvents with the jobTable, the result
>> will be the jobs that are not completed within time
>
> from expiredPublisherEvents as ep join jobTable as t
>> on ep.pid == t.pid
>> select ep.pid, t.sid
>> insert into *jobsNotCompletedInTime*;
>
>
>> -- now delete those uncompleted jobs from the jobTable as well
>> from jobsNotCompletedInTime
>> delete jobTable
>> on jobTable.pid == pid and jobTable.sid == sid;
>
>
>
> Hope that helped;
>
> Regards,
> Grainier.
>
>
> On Fri, Sep 29, 2017 at 11:17 AM, Jayesh Senjaliya 
> wrote:
>
>> Hi Gobinath,
>>
>> Thanks for the suggestion about absence pattern, but we wont be able to
>> upgrade to Siddhi 4 anytime soon.
>>
>> I am basically at the point where I can get all the relevant (subscribe)
>> events that happened during the given interval of first arrival of publish
>> events.
>>
>> here AllPublisher = all registered publisher-subscriber events ( for each
>> publisher event )
>>
>> from registeryTable as s join #publisher as p on p.pid == s.pid
>> select p.pid, s.sid insert into AllPublisher;
>>
>>
>> from every(a=AllPublisher) -> s=subscriber[pid == a.pid]
>> within 1 sec
>> select a.pid, s.sid
>> insert into completed_jobs_in_1_sec;
>>
>> now, i need to find out those events from publisher (a) that dint had any
>> match from s within 1 sec.
>>
>> which I was expecting to be available with " insert *expired events*
>> into not_completed_jobs", but looks like expired events are only available
>> when window is used. I am also looking at the code to see if i should add
>> this.
>>
>> Thanks
>> Jayesh
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Sep 27, 2017 at 4:21 AM, Gobinath  wrote:
>>
>>> Hi,
>>>
>>> If you can use Siddhi 4 snapshot release, it can be done using the new
>>> feature 'Absent Pattern' added to Siddhi 4. The query to detect the events
>>> that do not match the condition within 10 seconds is given below:
>>>
>>> from every e1=publisher -> not subscriber[e1.pid == pid] for 10 sec
>>> select e1.pid
>>> insert into not_completed_jobs_in_time;
>>>
>>> The above query waits for 10 seconds from the arrival of every publisher
>>> event and if there is no subscriber event with an id satisfying the
>>> condition arrived within that waiting period, the id of the publisher event
>>> will be inserted into the not_completed_jobs_in_time stream.
>>>
>>> I guess the official document for Siddhi 4 is under construction. So you
>>> can find more details about absent pattern at [1]
>>>
>>> Still, Siddhi 4 is not production ready so I wonder whether you can use
>>> this feature or not

Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Grainier Perera]

Hi Jay,

Since you are maintaining a jobTable, you can use that to identify
completed and non-completed jobs in a given time frame. Refer to the below
snippet;

@Plan:name('ExecutionPlan')


> @Import('subStream:1.0.0')
> define stream subStream (pid int, sid int, status string);


> @Import('pubStream:1.0.0')
> define stream pubStream (pid int, status string);


> @Export('jobsNotCompletedInTime:1.0.0')
> define stream jobsNotCompletedInTime (pid int, sid int);


> @Export('jobsCompletedInTime:1.0.0')
> define stream jobsCompletedInTime (pid int, sid int);


> @Export('foo:1.0.0')
> define stream foo (pid int, sid int);


> @From(eventtable='rdbms', jdbc.url='jdbc:mysql://localhost:3306/pubsub',
> username='root', password='root', driver.name='com.mysql.jdbc.Driver',
> table.name='jobTable')
> define table jobTable(pid int, sid int);


> from pubStream as p join jobTable as t
> on p.pid == t.pid
> select p.pid, t.sid
> insert into allPSJobs;



-- this will emit an event after 10 sec once a publish event arrives

from pubStream#window.time(10 sec)
> select *
> insert expired events into *expiredPublisherEvents*;



-- if the pattern matches within 10 sec, that means the job completed
> within time

from every (a=allPSJobs) -> s=subStream[pid==a.pid and sid == a.sid]
> within 10 sec
> select s.pid, s.sid
> insert into *jobsCompletedInTime*;


> -- delete completed jobs from the table, then what's left in the table are
> the uncompleted jobs
> from jobsCompletedInTime
> delete jobTable
> on jobTable.pid == pid and jobTable.sid == sid;


> -- so if we join expiredPublisherEvents with the jobTable, the result
> will be the jobs that are not completed within time

from expiredPublisherEvents as ep join jobTable as t
> on ep.pid == t.pid
> select ep.pid, t.sid
> insert into *jobsNotCompletedInTime*;


> -- now delete those uncompleted jobs from the jobTable as well
> from jobsNotCompletedInTime
> delete jobTable
> on jobTable.pid == pid and jobTable.sid == sid;



Hope that helped;

Regards,
Grainier.


On Fri, Sep 29, 2017 at 11:17 AM, Jayesh Senjaliya 
wrote:

> Hi Gobinath,
>
> Thanks for the suggestion about absence pattern, but we wont be able to
> upgrade to Siddhi 4 anytime soon.
>
> I am basically at the point where I can get all the relevant (subscribe)
> events that happened during the given interval of first arrival of publish
> events.
>
> here AllPublisher = all registered publisher-subscriber events ( for each
> publisher event )
>
> from registeryTable as s join #publisher as p on p.pid == s.pid
> select p.pid, s.sid insert into AllPublisher;
>
>
> from every(a=AllPublisher) -> s=subscriber[pid == a.pid]
> within 1 sec
> select a.pid, s.sid
> insert into completed_jobs_in_1_sec;
>
> now, i need to find out those events from publisher (a) that dint had any
> match from s within 1 sec.
>
> which I was expecting to be available with " insert *expired events* into
> not_completed_jobs", but looks like expired events are only available when
> window is used. I am also looking at the code to see if i should add this.
>
> Thanks
> Jayesh
>
>
>
>
>
>
>
>
>
> On Wed, Sep 27, 2017 at 4:21 AM, Gobinath  wrote:
>
>> Hi,
>>
>> If you can use Siddhi 4 snapshot release, it can be done using the new
>> feature 'Absent Pattern' added to Siddhi 4. The query to detect the events
>> that do not match the condition within 10 seconds is given below:
>>
>> from every e1=publisher -> not subscriber[e1.pid == pid] for 10 sec
>> select e1.pid
>> insert into not_completed_jobs_in_time;
>>
>> The above query waits for 10 seconds from the arrival of every publisher
>> event and if there is no subscriber event with an id satisfying the
>> condition arrived within that waiting period, the id of the publisher event
>> will be inserted into the not_completed_jobs_in_time stream.
>>
>> I guess the official document for Siddhi 4 is under construction. So you
>> can find more details about absent pattern at [1]
>>
>> Still, Siddhi 4 is not production ready so I wonder whether you can use
>> this feature or not.
>>
>> [1] http://www.javahelps.com/2017/08/detect-absence-of-events-ws
>> o2-siddhi.html
>>
>>
>>
>>
>> On Tue, Sep 26, 2017 at 10:05 PM, Jayesh Senjaliya 
>> wrote:
>>
>>> Hi Grainier,
>>>
>>> ya, i came across that example page, but i think that does not work in
>>> my use-case which is as follow.
>>>
>>> i have a publish event followed by multiple subscribe event for the same
>>> publish job.
>>> now i want to catch if certain jobs (publish -> subscribe) has been
>>> finished with 10 sec.
>>> I have all the registered jobs in db table, which i use to gather all
>>> the required publish-subscribe job events.
>>>
>>> define table jobTable( pid string, sid string);
>>> define stream pubStream (pid int, status string);
>>> define stream subStream (pid int, sid int, status string);
>>>
>>> -- this will get all the publish-> subscribe jobs events as master list
>>> from pubStream as p join jobTable as t
>>> on p.pid == 

Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Jayesh Senjaliya]

Hi Gobinath,

Thanks for the suggestion about absence pattern, but we wont be able to
upgrade to Siddhi 4 anytime soon.

I am basically at the point where I can get all the relevant (subscribe)
events that happened during the given interval of first arrival of publish
events.

here AllPublisher = all registered publisher-subscriber events ( for each
publisher event )

from registeryTable as s join #publisher as p on p.pid == s.pid
select p.pid, s.sid insert into AllPublisher;


from every(a=AllPublisher) -> s=subscriber[pid == a.pid]
within 1 sec
select a.pid, s.sid
insert into completed_jobs_in_1_sec;

now, i need to find out those events from publisher (a) that dint had any
match from s within 1 sec.

which I was expecting to be available with " insert *expired events* into
not_completed_jobs", but looks like expired events are only available when
window is used. I am also looking at the code to see if i should add this.

Thanks
Jayesh









On Wed, Sep 27, 2017 at 4:21 AM, Gobinath  wrote:

> Hi,
>
> If you can use Siddhi 4 snapshot release, it can be done using the new
> feature 'Absent Pattern' added to Siddhi 4. The query to detect the events
> that do not match the condition within 10 seconds is given below:
>
> from every e1=publisher -> not subscriber[e1.pid == pid] for 10 sec
> select e1.pid
> insert into not_completed_jobs_in_time;
>
> The above query waits for 10 seconds from the arrival of every publisher
> event and if there is no subscriber event with an id satisfying the
> condition arrived within that waiting period, the id of the publisher event
> will be inserted into the not_completed_jobs_in_time stream.
>
> I guess the official document for Siddhi 4 is under construction. So you
> can find more details about absent pattern at [1]
>
> Still, Siddhi 4 is not production ready so I wonder whether you can use
> this feature or not.
>
> [1] http://www.javahelps.com/2017/08/detect-absence-of-events-
> wso2-siddhi.html
>
>
>
>
> On Tue, Sep 26, 2017 at 10:05 PM, Jayesh Senjaliya 
> wrote:
>
>> Hi Grainier,
>>
>> ya, i came across that example page, but i think that does not work in my
>> use-case which is as follow.
>>
>> i have a publish event followed by multiple subscribe event for the same
>> publish job.
>> now i want to catch if certain jobs (publish -> subscribe) has been
>> finished with 10 sec.
>> I have all the registered jobs in db table, which i use to gather all the
>> required publish-subscribe job events.
>>
>> define table jobTable( pid string, sid string);
>> define stream pubStream (pid int, status string);
>> define stream subStream (pid int, sid int, status string);
>>
>> -- this will get all the publish-> subscribe jobs events as master list
>> from pubStream as p join jobTable as t
>> on p.pid == t.pid
>> select p.pid, t.sid insert into allPSJobs;
>>
>> -- this is where i need to do intersection where if subStream event is
>> seen within 2 sec then remove that from master list ( allPSJobs ) if not
>> include that in not_completed_jobs_in_time
>>
>> from every ( a=allPSJobs ) -> s= subStream[sid == a.sid and pid==a.pid ]
>> within 2 sec
>> select s.pid, s.sid insert into completed_jobs_in_time;
>>
>>
>> hope that make sense from what i am trying to do.
>>
>> Thanks
>> Jayesh
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Sep 25, 2017 at 8:39 AM, Grainier Perera 
>> wrote:
>>
>>> Hi Jay,
>>>
>>> You can try something similar to this to get non-matched events during
>>> last 10 secs; You can find some documentation on this as well; link
>>> 
>>>
>>>
>>>
 define stream publisher (pid string, time string);
 define stream subscriber (pid string, sid string, time string);
>>>
>>>
 from publisher#window.time(10 sec)
 select *
 insert expired events into expired_publisher;
>>>
>>>
 from every pub=publisher -> sub=subscriber[pub.pid == pid] or
 exp=expired_publisher[pub.pid == pid]
 select pub.pid as pid, pub.time as time, sub.pid as subPid
 insert into filter_stream;
>>>
>>>
 from filter_stream [(subPid is null)]
 select pid, time
 insert into not_seen_in_last_10_sec_events;
>>>
>>>
>>> Moreover, I didn't get what you meant by "also is there a way to perform
>>> intersection of events based on grouping or time window ?" can you please
>>> elaborate on this?
>>>
>>> Regards,
>>>
>>> On Mon, Sep 25, 2017 at 11:02 AM, Jayesh Senjaliya 
>>> wrote:
>>>
 Hi,

 is there a way to get events that didnt match within the given time
 frame.

 for example:

 define stream publisher (pid string, time string);
 define stream subscriber (pid string, sid string, time string);

 from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
 within 10 sec
 select e1.pid, e2.sid
 insert into seen_in_last_10_sec_events;


 so if i have matching event above, i will see it in
 seen_in_last_10_sec_e

Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Gobinath]

Hi,

If you can use Siddhi 4 snapshot release, it can be done using the new
feature 'Absent Pattern' added to Siddhi 4. The query to detect the events
that do not match the condition within 10 seconds is given below:

from every e1=publisher -> not subscriber[e1.pid == pid] for 10 sec
select e1.pid
insert into not_completed_jobs_in_time;

The above query waits for 10 seconds from the arrival of every publisher
event and if there is no subscriber event with an id satisfying the
condition arrived within that waiting period, the id of the publisher event
will be inserted into the not_completed_jobs_in_time stream.

I guess the official document for Siddhi 4 is under construction. So you
can find more details about absent pattern at [1]

Still, Siddhi 4 is not production ready so I wonder whether you can use
this feature or not.

[1]
http://www.javahelps.com/2017/08/detect-absence-of-events-wso2-siddhi.html




On Tue, Sep 26, 2017 at 10:05 PM, Jayesh Senjaliya 
wrote:

> Hi Grainier,
>
> ya, i came across that example page, but i think that does not work in my
> use-case which is as follow.
>
> i have a publish event followed by multiple subscribe event for the same
> publish job.
> now i want to catch if certain jobs (publish -> subscribe) has been
> finished with 10 sec.
> I have all the registered jobs in db table, which i use to gather all the
> required publish-subscribe job events.
>
> define table jobTable( pid string, sid string);
> define stream pubStream (pid int, status string);
> define stream subStream (pid int, sid int, status string);
>
> -- this will get all the publish-> subscribe jobs events as master list
> from pubStream as p join jobTable as t
> on p.pid == t.pid
> select p.pid, t.sid insert into allPSJobs;
>
> -- this is where i need to do intersection where if subStream event is
> seen within 2 sec then remove that from master list ( allPSJobs ) if not
> include that in not_completed_jobs_in_time
>
> from every ( a=allPSJobs ) -> s= subStream[sid == a.sid and pid==a.pid ]
> within 2 sec
> select s.pid, s.sid insert into completed_jobs_in_time;
>
>
> hope that make sense from what i am trying to do.
>
> Thanks
> Jayesh
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Sep 25, 2017 at 8:39 AM, Grainier Perera 
> wrote:
>
>> Hi Jay,
>>
>> You can try something similar to this to get non-matched events during
>> last 10 secs; You can find some documentation on this as well; link
>> 
>>
>>
>>
>>> define stream publisher (pid string, time string);
>>> define stream subscriber (pid string, sid string, time string);
>>
>>
>>> from publisher#window.time(10 sec)
>>> select *
>>> insert expired events into expired_publisher;
>>
>>
>>> from every pub=publisher -> sub=subscriber[pub.pid == pid] or
>>> exp=expired_publisher[pub.pid == pid]
>>> select pub.pid as pid, pub.time as time, sub.pid as subPid
>>> insert into filter_stream;
>>
>>
>>> from filter_stream [(subPid is null)]
>>> select pid, time
>>> insert into not_seen_in_last_10_sec_events;
>>
>>
>> Moreover, I didn't get what you meant by "also is there a way to perform
>> intersection of events based on grouping or time window ?" can you please
>> elaborate on this?
>>
>> Regards,
>>
>> On Mon, Sep 25, 2017 at 11:02 AM, Jayesh Senjaliya 
>> wrote:
>>
>>> Hi,
>>>
>>> is there a way to get events that didnt match within the given time
>>> frame.
>>>
>>> for example:
>>>
>>> define stream publisher (pid string, time string);
>>> define stream subscriber (pid string, sid string, time string);
>>>
>>> from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
>>> within 10 sec
>>> select e1.pid, e2.sid
>>> insert into seen_in_last_10_sec_events;
>>>
>>>
>>> so if i have matching event above, i will see it in
>>> seen_in_last_10_sec_events, but is there a way to get all events or non
>>> matched events during that last 10 seconds from publisher or subscriber ?
>>>
>>> also is there a way to perform intersection of events based on grouping
>>> or time window ?
>>>
>>>
>>> Thanks
>>> Jay
>>>
>>> ___
>>> Dev mailing list
>>> Dev@wso2.org
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>> Grainier Perera
>> Senior Software Engineer
>> Mobile : +94716122384 <+94%2071%20612%202384>
>> WSO2 Inc. | http://wso2.com
>> lean.enterprise.middleware
>>
>
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 
*Gobinath** Loganathan*
Graduate Student,
Electrical and Computer Engineering,
Western University.
Email  : slgobin...@gmail.com
Blog: javahelps.com 
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Jayesh Senjaliya]

sorry bit correction on the following line in *BOLD*.

.now i want to catch if certain jobs (publish -> subscribe) has *NOT* been
finished with 10 sec.


On Tue, Sep 26, 2017 at 7:05 PM, Jayesh Senjaliya 
wrote:

> Hi Grainier,
>
> ya, i came across that example page, but i think that does not work in my
> use-case which is as follow.
>
> i have a publish event followed by multiple subscribe event for the same
> publish job.
> now i want to catch if certain jobs (publish -> subscribe) has been
> finished with 10 sec.
> I have all the registered jobs in db table, which i use to gather all the
> required publish-subscribe job events.
>
> define table jobTable( pid string, sid string);
> define stream pubStream (pid int, status string);
> define stream subStream (pid int, sid int, status string);
>
> -- this will get all the publish-> subscribe jobs events as master list
> from pubStream as p join jobTable as t
> on p.pid == t.pid
> select p.pid, t.sid insert into allPSJobs;
>
> -- this is where i need to do intersection where if subStream event is
> seen within 2 sec then remove that from master list ( allPSJobs ) if not
> include that in not_completed_jobs_in_time
>
> from every ( a=allPSJobs ) -> s= subStream[sid == a.sid and pid==a.pid ]
> within 2 sec
> select s.pid, s.sid insert into completed_jobs_in_time;
>
>
> hope that make sense from what i am trying to do.
>
> Thanks
> Jayesh
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Sep 25, 2017 at 8:39 AM, Grainier Perera 
> wrote:
>
>> Hi Jay,
>>
>> You can try something similar to this to get non-matched events during
>> last 10 secs; You can find some documentation on this as well; link
>> 
>>
>>
>>
>>> define stream publisher (pid string, time string);
>>> define stream subscriber (pid string, sid string, time string);
>>
>>
>>> from publisher#window.time(10 sec)
>>> select *
>>> insert expired events into expired_publisher;
>>
>>
>>> from every pub=publisher -> sub=subscriber[pub.pid == pid] or
>>> exp=expired_publisher[pub.pid == pid]
>>> select pub.pid as pid, pub.time as time, sub.pid as subPid
>>> insert into filter_stream;
>>
>>
>>> from filter_stream [(subPid is null)]
>>> select pid, time
>>> insert into not_seen_in_last_10_sec_events;
>>
>>
>> Moreover, I didn't get what you meant by "also is there a way to perform
>> intersection of events based on grouping or time window ?" can you please
>> elaborate on this?
>>
>> Regards,
>>
>> On Mon, Sep 25, 2017 at 11:02 AM, Jayesh Senjaliya 
>> wrote:
>>
>>> Hi,
>>>
>>> is there a way to get events that didnt match within the given time
>>> frame.
>>>
>>> for example:
>>>
>>> define stream publisher (pid string, time string);
>>> define stream subscriber (pid string, sid string, time string);
>>>
>>> from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
>>> within 10 sec
>>> select e1.pid, e2.sid
>>> insert into seen_in_last_10_sec_events;
>>>
>>>
>>> so if i have matching event above, i will see it in
>>> seen_in_last_10_sec_events, but is there a way to get all events or non
>>> matched events during that last 10 seconds from publisher or subscriber ?
>>>
>>> also is there a way to perform intersection of events based on grouping
>>> or time window ?
>>>
>>>
>>> Thanks
>>> Jay
>>>
>>> ___
>>> Dev mailing list
>>> Dev@wso2.org
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>> Grainier Perera
>> Senior Software Engineer
>> Mobile : +94716122384 <+94%2071%20612%202384>
>> WSO2 Inc. | http://wso2.com
>> lean.enterprise.middleware
>>
>
>
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Jayesh Senjaliya]

Hi Grainier,

ya, i came across that example page, but i think that does not work in my
use-case which is as follow.

i have a publish event followed by multiple subscribe event for the same
publish job.
now i want to catch if certain jobs (publish -> subscribe) has been
finished with 10 sec.
I have all the registered jobs in db table, which i use to gather all the
required publish-subscribe job events.

define table jobTable( pid string, sid string);
define stream pubStream (pid int, status string);
define stream subStream (pid int, sid int, status string);

-- this will get all the publish-> subscribe jobs events as master list
from pubStream as p join jobTable as t
on p.pid == t.pid
select p.pid, t.sid insert into allPSJobs;

-- this is where i need to do intersection where if subStream event is seen
within 2 sec then remove that from master list ( allPSJobs ) if not include
that in not_completed_jobs_in_time

from every ( a=allPSJobs ) -> s= subStream[sid == a.sid and pid==a.pid ]
within 2 sec
select s.pid, s.sid insert into completed_jobs_in_time;


hope that make sense from what i am trying to do.

Thanks
Jayesh











On Mon, Sep 25, 2017 at 8:39 AM, Grainier Perera  wrote:

> Hi Jay,
>
> You can try something similar to this to get non-matched events during
> last 10 secs; You can find some documentation on this as well; link
> 
>
>
>
>> define stream publisher (pid string, time string);
>> define stream subscriber (pid string, sid string, time string);
>
>
>> from publisher#window.time(10 sec)
>> select *
>> insert expired events into expired_publisher;
>
>
>> from every pub=publisher -> sub=subscriber[pub.pid == pid] or
>> exp=expired_publisher[pub.pid == pid]
>> select pub.pid as pid, pub.time as time, sub.pid as subPid
>> insert into filter_stream;
>
>
>> from filter_stream [(subPid is null)]
>> select pid, time
>> insert into not_seen_in_last_10_sec_events;
>
>
> Moreover, I didn't get what you meant by "also is there a way to perform
> intersection of events based on grouping or time window ?" can you please
> elaborate on this?
>
> Regards,
>
> On Mon, Sep 25, 2017 at 11:02 AM, Jayesh Senjaliya 
> wrote:
>
>> Hi,
>>
>> is there a way to get events that didnt match within the given time frame.
>>
>> for example:
>>
>> define stream publisher (pid string, time string);
>> define stream subscriber (pid string, sid string, time string);
>>
>> from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
>> within 10 sec
>> select e1.pid, e2.sid
>> insert into seen_in_last_10_sec_events;
>>
>>
>> so if i have matching event above, i will see it in
>> seen_in_last_10_sec_events, but is there a way to get all events or non
>> matched events during that last 10 seconds from publisher or subscriber ?
>>
>> also is there a way to perform intersection of events based on grouping
>> or time window ?
>>
>>
>> Thanks
>> Jay
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> Grainier Perera
> Senior Software Engineer
> Mobile : +94716122384 <+94%2071%20612%202384>
> WSO2 Inc. | http://wso2.com
> lean.enterprise.middleware
>
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Grainier Perera]

Hi Jay,

You can try something similar to this to get non-matched events during last
10 secs; You can find some documentation on this as well; link




> define stream publisher (pid string, time string);
> define stream subscriber (pid string, sid string, time string);


> from publisher#window.time(10 sec)
> select *
> insert expired events into expired_publisher;


> from every pub=publisher -> sub=subscriber[pub.pid == pid] or
> exp=expired_publisher[pub.pid == pid]
> select pub.pid as pid, pub.time as time, sub.pid as subPid
> insert into filter_stream;


> from filter_stream [(subPid is null)]
> select pid, time
> insert into not_seen_in_last_10_sec_events;


Moreover, I didn't get what you meant by "also is there a way to perform
intersection of events based on grouping or time window ?" can you please
elaborate on this?

Regards,

On Mon, Sep 25, 2017 at 11:02 AM, Jayesh Senjaliya 
wrote:

> Hi,
>
> is there a way to get events that didnt match within the given time frame.
>
> for example:
>
> define stream publisher (pid string, time string);
> define stream subscriber (pid string, sid string, time string);
>
> from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
> within 10 sec
> select e1.pid, e2.sid
> insert into seen_in_last_10_sec_events;
>
>
> so if i have matching event above, i will see it in
> seen_in_last_10_sec_events, but is there a way to get all events or non
> matched events during that last 10 seconds from publisher or subscriber ?
>
> also is there a way to perform intersection of events based on grouping or
> time window ?
>
>
> Thanks
> Jay
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 
Grainier Perera
Senior Software Engineer
Mobile : +94716122384
WSO2 Inc. | http://wso2.com
lean.enterprise.middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [Siddhi] non-matched or expired events in pattern query

[Jayesh Senjaliya]

Hi,

is there a way to get events that didnt match within the given time frame.

for example:

define stream publisher (pid string, time string);
define stream subscriber (pid string, sid string, time string);

from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
within 10 sec
select e1.pid, e2.sid
insert into seen_in_last_10_sec_events;


so if i have matching event above, i will see it in
seen_in_last_10_sec_events, but is there a way to get all events or non
matched events during that last 10 seconds from publisher or subscriber ?

also is there a way to perform intersection of events based on grouping or
time window ?


Thanks
Jay
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev