Re: [Dev] [Siddhi] Incorrect results when having two window joins with sums
Hi all,
Thanks Tishan for the explanation.
I have used step 1 and 2 with "join with window.length(1) of each stream"
and Im getting the results as expected.
Best Regards,
Ramindu.
On Sun, Oct 28, 2018 at 11:18 AM Sriskandarajah Suhothayan
wrote:
>
>
> On Wed, Oct 17, 2018 at 11:59 PM Tishan Dahanayakage
> wrote:
>
>> Hi Ramindu,
>>
>> Each time a join happens s.amount is populated as 100 and it is a current
>> event. When that current event reach the sum attribute aggregator it will
>> keep on adding. In other words sum(s.amount) represent the sum of amounts
>> that joined with consumptions stream not sum of amount came within last
>> hour.
>>
>> If we are to accurately achieve this requirement we should take below
>> approach.
>> 1) Window query to calculate running sum of last hour of supply stream
>> 2) Window query to calculate running sum of last hour of consumption
>> stream
>> 3) Pattern query to compare each supply with proceeding consumptions
>> which fulfills the conditions.
>>
>> +1
> #3 can also be a join with window.length(1) of each stream, just to join
> and compare .
>
> Thanks,
>> /Tishan
>>
>> On Wed, Oct 17, 2018 at 10:46 PM Ramindu De Silva
>> wrote:
>>
>>> Hi all,
>>>
>>> In tutorial[1] which we are using for our labkit as well has the siddhi
>>> app as follows.
>>>
>>> @App:name('MaterialThresholdAlertApp')
>>>
>>> @source(type = 'http', @map(type = 'json'))
>>> define stream MaterialConsumptionStream(name string, user string, amount
>>> double);
>>>
>>> @source(type = 'http', @map(type = 'json'))
>>> define stream MaterialSupplyStream(name string, supplier string, amount
>>> double);
>>>
>>> @sink(type='log', prefix='Materials that go beyond sustainability
>>> threshold:')
>>> define stream MaterialThresholdAlertStream(name string, supplyAmount
>>> double, consumptionAmount double, user string, supplier string);
>>>
>>> from MaterialConsumptionStream#window.time(1 hour) as c
>>> join MaterialSupplyStream#window.time(1 hour) as s
>>> on c.name == s.name
>>> select s.name, s.amount as supplyAmount, c.amount as consumptionAmount,
>>> user, supplier
>>> group by s.name
>>> having s.amount * 0.95 < c.amount
>>> insert into MaterialThresholdAlertStream;
>>>
>>>
>>> But in-order to check the total consumed amount is greater than 95% of
>>> supplied amount, we should have the query as follows.
>>>
>>> select s.name, sum(s.amount) as supplyAmount, sum(c.amount) as
>>> consumptionAmount, user, supplier
>>>
>>>
>>> But the results gets printed as follows when we simulate using the
>>> following steps,
>>>
>>>1. Simulate "*MaterialSupplyStream*" with "sugar, yyy, 100"
>>>2. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
>>> 1. Materials that go beyond sustainability threshold: :
>>> Event{timestamp=1539794935863, data=[sugar, *100.0*, 97.0, yyy,
>>> xxx], isExpired=false}
>>> 3. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
>>> 1. Materials that go beyond sustainability threshold: :
>>> Event{timestamp=1539794936733, data=[sugar, *200.0*, 194.0, yyy,
>>> xxx], isExpired=false}
>>> 4. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
>>> 1. Materials that go beyond sustainability threshold: :
>>> Event{timestamp=1539794937643, data=[sugar, *300.0*, 291.0, yyy,
>>> xxx], isExpired=false}
>>>
>>> Even though we dont send event to the *MaterialSupplyStream,* the
>>> summation adds 100 each at each join. It should be resulted with the
>>> initial 100 that we sent. Is it? Please correct me if I'm wrong.
>>>
>>> Best Regards,
>>> Ramindu.
>>>
>>> 1. https://docs.wso2.com/display/SP430/Correlating+Simple+Events
>>>
>>>
>>> --
>>> *Ramindu De Silva*
>>> Senior Software Engineer
>>> WSO2 Inc.: http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> email: ramin...@wso2.com
>>> mob: +94 719678895
>>>
>>
>>
>> --
>> *Tishan Dahanayakage* | Associate Technical Lead | WSO2 Inc.
>> (m) +94716481328 | (w) +94112145345 | (e) tis...@wso2.com
>> GET INTEGRATION AGILE
>> Integration Agility for Digitally Driven Business
>>
>> Disclaimer: This communication may contain privileged or other
>> confidential information and is intended exclusively for the addressee/s.
>> If you are not the intended recipient/s, or believe that you may have
>> received this communication in error, please reply to the sender indicating
>> that fact and delete the copy you received and in addition, you should not
>> print, copy, re-transmit, disseminate, or otherwise use the information
>> contained in this communication. Internet communications cannot be
>> guaranteed to be timely, secure, error or virus-free. The sender does not
>> accept liability for any errors or omissions.
>>
>
>
> --
> *S. Suhothayan* | Director | WSO2 Inc.
> GET INTEGRATION AGILE
> Integration Agility for Digitally
Re: [Dev] [Siddhi] Incorrect results when having two window joins with sums
On Wed, Oct 17, 2018 at 11:59 PM Tishan Dahanayakage
wrote:
> Hi Ramindu,
>
> Each time a join happens s.amount is populated as 100 and it is a current
> event. When that current event reach the sum attribute aggregator it will
> keep on adding. In other words sum(s.amount) represent the sum of amounts
> that joined with consumptions stream not sum of amount came within last
> hour.
>
> If we are to accurately achieve this requirement we should take below
> approach.
> 1) Window query to calculate running sum of last hour of supply stream
> 2) Window query to calculate running sum of last hour of consumption
> stream
> 3) Pattern query to compare each supply with proceeding consumptions which
> fulfills the conditions.
>
> +1
#3 can also be a join with window.length(1) of each stream, just to join
and compare .
Thanks,
> /Tishan
>
> On Wed, Oct 17, 2018 at 10:46 PM Ramindu De Silva
> wrote:
>
>> Hi all,
>>
>> In tutorial[1] which we are using for our labkit as well has the siddhi
>> app as follows.
>>
>> @App:name('MaterialThresholdAlertApp')
>>
>> @source(type = 'http', @map(type = 'json'))
>> define stream MaterialConsumptionStream(name string, user string, amount
>> double);
>>
>> @source(type = 'http', @map(type = 'json'))
>> define stream MaterialSupplyStream(name string, supplier string, amount
>> double);
>>
>> @sink(type='log', prefix='Materials that go beyond sustainability
>> threshold:')
>> define stream MaterialThresholdAlertStream(name string, supplyAmount double,
>> consumptionAmount double, user string, supplier string);
>>
>> from MaterialConsumptionStream#window.time(1 hour) as c
>> join MaterialSupplyStream#window.time(1 hour) as s
>> on c.name == s.name
>> select s.name, s.amount as supplyAmount, c.amount as consumptionAmount,
>> user, supplier
>> group by s.name
>> having s.amount * 0.95 < c.amount
>> insert into MaterialThresholdAlertStream;
>>
>>
>> But in-order to check the total consumed amount is greater than 95% of
>> supplied amount, we should have the query as follows.
>>
>> select s.name, sum(s.amount) as supplyAmount, sum(c.amount) as
>> consumptionAmount, user, supplier
>>
>>
>> But the results gets printed as follows when we simulate using the
>> following steps,
>>
>>1. Simulate "*MaterialSupplyStream*" with "sugar, yyy, 100"
>>2. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
>> 1. Materials that go beyond sustainability threshold: :
>> Event{timestamp=1539794935863, data=[sugar, *100.0*, 97.0, yyy,
>> xxx], isExpired=false}
>> 3. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
>> 1. Materials that go beyond sustainability threshold: :
>> Event{timestamp=1539794936733, data=[sugar, *200.0*, 194.0, yyy,
>> xxx], isExpired=false}
>> 4. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
>> 1. Materials that go beyond sustainability threshold: :
>> Event{timestamp=1539794937643, data=[sugar, *300.0*, 291.0, yyy,
>> xxx], isExpired=false}
>>
>> Even though we dont send event to the *MaterialSupplyStream,* the
>> summation adds 100 each at each join. It should be resulted with the
>> initial 100 that we sent. Is it? Please correct me if I'm wrong.
>>
>> Best Regards,
>> Ramindu.
>>
>> 1. https://docs.wso2.com/display/SP430/Correlating+Simple+Events
>>
>>
>> --
>> *Ramindu De Silva*
>> Senior Software Engineer
>> WSO2 Inc.: http://wso2.com
>> lean.enterprise.middleware
>>
>> email: ramin...@wso2.com
>> mob: +94 719678895
>>
>
>
> --
> *Tishan Dahanayakage* | Associate Technical Lead | WSO2 Inc.
> (m) +94716481328 | (w) +94112145345 | (e) tis...@wso2.com
> GET INTEGRATION AGILE
> Integration Agility for Digitally Driven Business
>
> Disclaimer: This communication may contain privileged or other
> confidential information and is intended exclusively for the addressee/s.
> If you are not the intended recipient/s, or believe that you may have
> received this communication in error, please reply to the sender indicating
> that fact and delete the copy you received and in addition, you should not
> print, copy, re-transmit, disseminate, or otherwise use the information
> contained in this communication. Internet communications cannot be
> guaranteed to be timely, secure, error or virus-free. The sender does not
> accept liability for any errors or omissions.
>
--
*S. Suhothayan* | Director | WSO2 Inc.
GET INTEGRATION AGILE
Integration Agility for Digitally Driven Business
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [Siddhi] Incorrect results when having two window joins with sums
Hi Ramindu,
Each time a join happens s.amount is populated as 100 and it is a current
event. When that current event reach the sum attribute aggregator it will
keep on adding. In other words sum(s.amount) represent the sum of amounts
that joined with consumptions stream not sum of amount came within last
hour.
If we are to accurately achieve this requirement we should take below
approach.
1) Window query to calculate running sum of last hour of supply stream
2) Window query to calculate running sum of last hour of consumption stream
3) Pattern query to compare each supply with proceeding consumptions which
fulfills the conditions.
Thanks,
/Tishan
On Wed, Oct 17, 2018 at 10:46 PM Ramindu De Silva wrote:
> Hi all,
>
> In tutorial[1] which we are using for our labkit as well has the siddhi
> app as follows.
>
> @App:name('MaterialThresholdAlertApp')
>
> @source(type = 'http', @map(type = 'json'))
> define stream MaterialConsumptionStream(name string, user string, amount
> double);
>
> @source(type = 'http', @map(type = 'json'))
> define stream MaterialSupplyStream(name string, supplier string, amount
> double);
>
> @sink(type='log', prefix='Materials that go beyond sustainability threshold:')
> define stream MaterialThresholdAlertStream(name string, supplyAmount double,
> consumptionAmount double, user string, supplier string);
>
> from MaterialConsumptionStream#window.time(1 hour) as c
> join MaterialSupplyStream#window.time(1 hour) as s
> on c.name == s.name
> select s.name, s.amount as supplyAmount, c.amount as consumptionAmount, user,
> supplier
> group by s.name
> having s.amount * 0.95 < c.amount
> insert into MaterialThresholdAlertStream;
>
>
> But in-order to check the total consumed amount is greater than 95% of
> supplied amount, we should have the query as follows.
>
> select s.name, sum(s.amount) as supplyAmount, sum(c.amount) as
> consumptionAmount, user, supplier
>
>
> But the results gets printed as follows when we simulate using the
> following steps,
>
>1. Simulate "*MaterialSupplyStream*" with "sugar, yyy, 100"
>2. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
> 1. Materials that go beyond sustainability threshold: :
> Event{timestamp=1539794935863, data=[sugar, *100.0*, 97.0, yyy,
> xxx], isExpired=false}
> 3. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
> 1. Materials that go beyond sustainability threshold: :
> Event{timestamp=1539794936733, data=[sugar, *200.0*, 194.0, yyy,
> xxx], isExpired=false}
> 4. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
> 1. Materials that go beyond sustainability threshold: :
> Event{timestamp=1539794937643, data=[sugar, *300.0*, 291.0, yyy,
> xxx], isExpired=false}
>
> Even though we dont send event to the *MaterialSupplyStream,* the
> summation adds 100 each at each join. It should be resulted with the
> initial 100 that we sent. Is it? Please correct me if I'm wrong.
>
> Best Regards,
> Ramindu.
>
> 1. https://docs.wso2.com/display/SP430/Correlating+Simple+Events
>
>
> --
> *Ramindu De Silva*
> Senior Software Engineer
> WSO2 Inc.: http://wso2.com
> lean.enterprise.middleware
>
> email: ramin...@wso2.com
> mob: +94 719678895
>
--
*Tishan Dahanayakage* | Associate Technical Lead | WSO2 Inc.
(m) +94716481328 | (w) +94112145345 | (e) tis...@wso2.com
GET INTEGRATION AGILE
Integration Agility for Digitally Driven Business
Disclaimer: This communication may contain privileged or other confidential
information and is intended exclusively for the addressee/s. If you are not
the intended recipient/s, or believe that you may have received this
communication in error, please reply to the sender indicating that fact and
delete the copy you received and in addition, you should not print, copy,
re-transmit, disseminate, or otherwise use the information contained in
this communication. Internet communications cannot be guaranteed to be
timely, secure, error or virus-free. The sender does not accept liability
for any errors or omissions.
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[Dev] [Siddhi] Incorrect results when having two window joins with sums
Hi all,
In tutorial[1] which we are using for our labkit as well has the siddhi app
as follows.
@App:name('MaterialThresholdAlertApp')
@source(type = 'http', @map(type = 'json'))
define stream MaterialConsumptionStream(name string, user string,
amount double);
@source(type = 'http', @map(type = 'json'))
define stream MaterialSupplyStream(name string, supplier string, amount double);
@sink(type='log', prefix='Materials that go beyond sustainability threshold:')
define stream MaterialThresholdAlertStream(name string, supplyAmount
double, consumptionAmount double, user string, supplier string);
from MaterialConsumptionStream#window.time(1 hour) as c
join MaterialSupplyStream#window.time(1 hour) as s
on c.name == s.name
select s.name, s.amount as supplyAmount, c.amount as
consumptionAmount, user, supplier
group by s.name
having s.amount * 0.95 < c.amount
insert into MaterialThresholdAlertStream;
But in-order to check the total consumed amount is greater than 95% of
supplied amount, we should have the query as follows.
select s.name, sum(s.amount) as supplyAmount, sum(c.amount) as
consumptionAmount, user, supplier
But the results gets printed as follows when we simulate using the
following steps,
1. Simulate "*MaterialSupplyStream*" with "sugar, yyy, 100"
2. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
1. Materials that go beyond sustainability threshold: :
Event{timestamp=1539794935863, data=[sugar, *100.0*, 97.0, yyy, xxx],
isExpired=false}
3. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
1. Materials that go beyond sustainability threshold: :
Event{timestamp=1539794936733, data=[sugar, *200.0*, 194.0, yyy,
xxx], isExpired=false}
4. Simulate "*MaterialConsumptionStream*" with "sugar, xxx, 97"
1. Materials that go beyond sustainability threshold: :
Event{timestamp=1539794937643, data=[sugar, *300.0*, 291.0, yyy,
xxx], isExpired=false}
Even though we dont send event to the *MaterialSupplyStream,* the summation
adds 100 each at each join. It should be resulted with the initial 100 that
we sent. Is it? Please correct me if I'm wrong.
Best Regards,
Ramindu.
1. https://docs.wso2.com/display/SP430/Correlating+Simple+Events
--
*Ramindu De Silva*
Senior Software Engineer
WSO2 Inc.: http://wso2.com
lean.enterprise.middleware
email: ramin...@wso2.com
mob: +94 719678895
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
