Hi Jayesh,
I don't think the OOTB event windows will support the feature that you are
looking for. The best approach for you is to use an event table instead.
You try something similar to below;
@Plan:name('TestExecutionPlan')
> define stream jobsStream (id int, eventdata string);
> define table jobsTable (id int, eventdata string, arrival_timestamp long,
> event_timestamp long);
> define trigger dailyTrigger at '0 0 3 ? * * *';
> /* Introduce additional attributes, to be used with the logic.
> */
> from jobsStream
> select
> id,
> eventdata,
> time:timestampInMilliseconds() as arrival_timestamp,
> time:extract(time:timestampInMilliseconds(), 'HOUR') as arrival_hour,
> time:timestampInMilliseconds(eventdata) as event_timestamp,
> time:extract('HOUR', eventdata, 'yyyy-MM-dd HH:mm:ss.SSS') as event_hour
> insert into enrichedJobsStream;
> /* Filter events arrives between 00:00 to 03:00 (exclusive)
> */
> from enrichedJobsStream[arrival_hour >= 0 AND arrival_hour < 3]
> insert into candidateJobsStream;
> /* Assuming, if the event_hour >= 3 (events which came in the past day
> between 3 - midnight),
> * Update the table with the highest event_timestamp.
> * So that the table will have one entry per id with the highest
> event_timestamp.
> * These are the events which doesn't match the criteria.
> */
> from candidateJobsStream[event_hour >= 3]
> select id, eventdata, arrival_timestamp, event_timestamp
> insert overwrite jobsTable
> on jobsTable.id == id and jobsTable.event_timestamp < event_timestamp;
>
/* Once the trigger get fired at 03:00 Join trigger with the table,
> * at this time table will only have events that doesn't match the
> criteria.
> */
> from dailyTrigger join jobsTable
> select jobsTable.id, jobsTable.eventdata
> insert into criteriaNotMatchedStream;
> /* If the event_hour < 3 (events which came between midnight - 3),
> * Which means it satisfies the criteria.
> */
> from candidateJobsStream[event_hour < 3]
> select id, eventdata
> insert into criteriaMatchedStream;
> /* Clean the table, so that it'll be empty before processing it next
> midnight.
> */
> from criteriaMatchedStream
> delete jobsTable
> on jobsTable.id == id;
> from criteriaNotMatchedStream
> delete jobsTable
> on jobsTable.id == id;
Regards,
Grainier.
On Sat, Oct 7, 2017 at 5:22 AM, Jayesh Senjaliya <jayesh.senjal...@gmail.com
> wrote:
> Hi Grainier,
>
> I have following requirement
>
> job typically runs multiple time a day, but for all the jobs that runs
> between midnight to 3AM, there has to be a job that has eventdata >
> midnight,
>
> so want to detect an event which satisfy the condition, or create and
> alert event at 3AM that says there were no event matching that criteria.
>
>
> define stream Jobs (id int, arrival_time string, eventdata string);
>
> from Jobs
> select id, eventdata
> having time:timestampInMilliseconds(eventdata) -
> time:timestampInMilliseconds('2017-10-03 00:00:00.000') > 0
> insert into test;
>
>
>
> sample data :
>
> Jobs=[1,2017-10-03 00:01:59.000,2017-10-02 21:01:59.000]
> Jobs=[1,2017-10-03 00:02:15.000,2017-10-02 21:30:59.000]
> Jobs=[1,2017-10-03 00:02:30.000,2017-10-03 00:15:00.000]
> Jobs=[1,2017-10-03 00:02:58.000,2017-10-03 01:30:59.000]
>
> delay(1000)
> Jobs=[2,2017-10-03 00:00:50.000,2017-10-02 20:01:59.000]
> Jobs=[2,2017-10-03 00:01:15.000,2017-10-02 21:30:00.000]
> Jobs=[2,2017-10-03 00:02:30.000,2017-10-02 21:30:00.000]
> Jobs=[2,2017-10-03 00:02:40.000,2017-10-02 21:30:00.000]
>
> delay(1000)
> Jobs=[3,2017-10-03 00:00:50.000,2017-10-02 22:01:59.000]
> Jobs=[3,2017-10-03 00:01:15.000,2017-10-02 22:30:59.000]
> Jobs=[3,2017-10-03 00:02:30.000,2017-10-03 01:40:00.000]
> Jobs=[3,2017-10-03 00:02:40.000,2017-10-03 02:30:59.000]
>
>
>
> the output should be
> event with id : 1 and 3 met the requirement, but event with id 2 didnt.
>
>
>
> i was trying this with window.cron and then grouping it by id, but that
> didnt work.
>
> do you have an idea on who this can be achive?
>
> basically i douldnt find anything that will keep giving me events between
> midnight to 3 and then also collective result at the end of the window
> which is 3AM here.
>
> logically i want to do following.
>
> - start collecting events at midnight, filter it further for events having
> eventdata > midnight, if there is an event matching that criteria, create
> an event in a stream.
> - at 3AM i want to look at all those collected events and see if there is
> any event that has not matched the criteria at all and send an event with
> the id to another stream
>
>
> Thanks
> Jayesh
>
>
>
>
>
>
>
>
>
>
>
>
>
>
--
Grainier Perera
Senior Software Engineer
Mobile : +94716122384
WSO2 Inc. | http://wso2.com
lean.enterprise.middleware
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
