Re: [rules-users] Drools Fusion Dropping Events?
Last week I found and fixed a very bad bug that seems to fit your case
https://issues.jboss.org/browse/DROOLS-131
This code should work in 5.6.0-SNAPSHOT and later versions.
I have seen DROOLS-138, but won't be able to work on it before the weekend
Davide
On 05/15/2013 11:12 PM, Jason Barto wrote:
> Wolfgang,
> the code to reproduce is below. I'm hoping to process between 20k and
> 50k events through Drools per second thus the extreme high-throughput
> testing. I could settle for a single Drools node handling only say 5K
> per second IFF I could cluster Drools but I've not yet found a way to
> distribute workload across an active-active Drools cluster (seems
> there is no such thing?).
>
> Since you're recommendation I've shifted to using Drools 5.3 just FYI:
>
>
> ### Average.java ###
> package drools53fusioneval;
>
> import java.io.IOException;
> import java.util.Random;
> import org.drools.KnowledgeBase;
> import org.drools.KnowledgeBaseConfiguration;
> import org.drools.KnowledgeBaseFactory;
> import org.drools.builder.KnowledgeBuilder;
> import org.drools.builder.KnowledgeBuilderFactory;
> import org.drools.builder.ResourceType;
> import org.drools.conf.EventProcessingOption;
> import org.drools.io.ResourceFactory;
> import org.drools.runtime.StatefulKnowledgeSession;
> import org.drools.runtime.rule.WorkingMemoryEntryPoint;
>
> class AvgDFEChannel implements org.drools.runtime.Channel {
>
> @Override
> public void send(Object o) {
> System.err.println("Recieved channel message: " + o);
> }
> }
>
> public class Average {
>
> public static void main(String[] args) throws
> InterruptedException, IOException {
> KnowledgeBaseConfiguration kbconfig =
> KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
> kbconfig.setOption(EventProcessingOption.STREAM);
>
> KnowledgeBase kbase =
> KnowledgeBaseFactory.newKnowledgeBase(kbconfig);
>
> KnowledgeBuilder kbuilder =
> KnowledgeBuilderFactory.newKnowledgeBuilder();
>
> kbuilder.add(ResourceFactory.newClassPathResource("drools53fusioneval/basic.drl"),
> ResourceType.DRL);
> if (kbuilder.hasErrors()) {
> System.err.println(kbuilder.getErrors().toString());
> }
> kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
>
> final StatefulKnowledgeSession session =
> kbase.newStatefulKnowledgeSession();
> session.registerChannel("heartbeat", new AvgDFEChannel());
> WorkingMemoryEntryPoint ep01 =
> session.getWorkingMemoryEntryPoint("ep01");
>
> new Thread() {
> public void run() {
> session.fireUntilHalt();
> }
> }.start();
>
> Thread.sleep(5000); // give the engine time to get setup
>
> Server hiwaesdk = new Server("hiwaesdk");
> session.insert(hiwaesdk);
>
> long LIMIT = 1;
> long sentCount = 0;
> int batchSize = 1;
> Random rnd = new Random(System.nanoTime());
> int temp = 0;
> long startTS = System.nanoTime();
> while (sentCount < LIMIT) {
> for (int i = 0; i < batchSize; i++) {
> temp = rnd.nextInt(212);
> IntEvent evt = new IntEvent (temp);
> ep01.insert(evt);
> sentCount++;
> }
> Thread.sleep (0x1);
> }
> double duration = (System.nanoTime() - startTS)/100.0;
> System.out.println(LIMIT +" events generated in "+ duration +"
> milliseconds");
> System.out.println("Last temperature submitted: "+ temp);
> for (int i = 0; i < 5; i++) {
> System.out.println ("Sec "+ i +": "+ hiwaesdk.currentTemp
> +", "+ hiwaesdk.readingCount);
> Thread.sleep (1000);
> }
> System.exit(0);
> }
> }
>
>
> ### basic.drl ###
> package drools53fusioneval
>
> declare IntEvent
> @role ( event )
> end
>
> rule "number rule"
> when
> $e : IntEvent () from entry-point ep01
> $s : Server (hostname == "hiwaesdk")
> then
> $s.currentTemp = $e.data;
> $s.readingCount++;
> end
>
>
>
>
> On Thu, May 16, 2013 at 4:25 AM, Wolfgang Laun
> mailto:wolfgang.l...@gmail.com>> wrote:
>
> What you describe is definitely not "expected behaviour" - it's more
> like that race condition (or another one) already being in 5.3.
> Posting
> your code might be a good idea ;-)
> -W
>
> On 16/05/2013, Jason Barto > wrote:
> > I've been working to load test Drools 5.3 to determine if its a
> fit for a
> > project. As part of the test I programmatically insert events
> as rapidly
> > as possible; an example, my earlier test inserted 10k events in
> about
> > 300ms. There is currently a single rule which reads the event
> and stores
> > the event's value into the only fact in Drools. I'm very happy
> to report,
> > and I'm sure i
Re: [rules-users] Drools Fusion Dropping Events?
Wolfgang,
the code to reproduce is below. I'm hoping to process between 20k and 50k
events through Drools per second thus the extreme high-throughput testing.
I could settle for a single Drools node handling only say 5K per second IFF
I could cluster Drools but I've not yet found a way to distribute workload
across an active-active Drools cluster (seems there is no such thing?).
Since you're recommendation I've shifted to using Drools 5.3 just FYI:
### Average.java ###
package drools53fusioneval;
import java.io.IOException;
import java.util.Random;
import org.drools.KnowledgeBase;
import org.drools.KnowledgeBaseConfiguration;
import org.drools.KnowledgeBaseFactory;
import org.drools.builder.KnowledgeBuilder;
import org.drools.builder.KnowledgeBuilderFactory;
import org.drools.builder.ResourceType;
import org.drools.conf.EventProcessingOption;
import org.drools.io.ResourceFactory;
import org.drools.runtime.StatefulKnowledgeSession;
import org.drools.runtime.rule.WorkingMemoryEntryPoint;
class AvgDFEChannel implements org.drools.runtime.Channel {
@Override
public void send(Object o) {
System.err.println("Recieved channel message: " + o);
}
}
public class Average {
public static void main(String[] args) throws InterruptedException,
IOException {
KnowledgeBaseConfiguration kbconfig =
KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
kbconfig.setOption(EventProcessingOption.STREAM);
KnowledgeBase kbase =
KnowledgeBaseFactory.newKnowledgeBase(kbconfig);
KnowledgeBuilder kbuilder =
KnowledgeBuilderFactory.newKnowledgeBuilder();
kbuilder.add(ResourceFactory.newClassPathResource("drools53fusioneval/basic.drl"),
ResourceType.DRL);
if (kbuilder.hasErrors()) {
System.err.println(kbuilder.getErrors().toString());
}
kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
final StatefulKnowledgeSession session =
kbase.newStatefulKnowledgeSession();
session.registerChannel("heartbeat", new AvgDFEChannel());
WorkingMemoryEntryPoint ep01 =
session.getWorkingMemoryEntryPoint("ep01");
new Thread() {
public void run() {
session.fireUntilHalt();
}
}.start();
Thread.sleep(5000); // give the engine time to get setup
Server hiwaesdk = new Server("hiwaesdk");
session.insert(hiwaesdk);
long LIMIT = 1;
long sentCount = 0;
int batchSize = 1;
Random rnd = new Random(System.nanoTime());
int temp = 0;
long startTS = System.nanoTime();
while (sentCount < LIMIT) {
for (int i = 0; i < batchSize; i++) {
temp = rnd.nextInt(212);
IntEvent evt = new IntEvent (temp);
ep01.insert(evt);
sentCount++;
}
Thread.sleep (0x1);
}
double duration = (System.nanoTime() - startTS)/100.0;
System.out.println(LIMIT +" events generated in "+ duration +"
milliseconds");
System.out.println("Last temperature submitted: "+ temp);
for (int i = 0; i < 5; i++) {
System.out.println ("Sec "+ i +": "+ hiwaesdk.currentTemp +",
"+ hiwaesdk.readingCount);
Thread.sleep (1000);
}
System.exit(0);
}
}
### basic.drl ###
package drools53fusioneval
declare IntEvent
@role ( event )
end
rule "number rule"
when
$e : IntEvent () from entry-point ep01
$s : Server (hostname == "hiwaesdk")
then
$s.currentTemp = $e.data;
$s.readingCount++;
end
On Thu, May 16, 2013 at 4:25 AM, Wolfgang Laun wrote:
> What you describe is definitely not "expected behaviour" - it's more
> like that race condition (or another one) already being in 5.3. Posting
> your code might be a good idea ;-)
> -W
>
> On 16/05/2013, Jason Barto wrote:
> > I've been working to load test Drools 5.3 to determine if its a fit for a
> > project. As part of the test I programmatically insert events as rapidly
> > as possible; an example, my earlier test inserted 10k events in about
> > 300ms. There is currently a single rule which reads the event and stores
> > the event's value into the only fact in Drools. I'm very happy to report,
> > and I'm sure it will be no surprise to anyone, that the engine processes
> > all the events in roughly 1 sec. However I have noticed that any large
> > number of events (~>1000) usually sees that a small number of events
> don't
> > get processed. I think after 10k events as many as 7 appear to have gone
> > unprocessed. If 100 events are inserted, rather than 10k, no events get
> > disregarded. Being new to Drools I can easily accept that my java code
> or
> > DRL code is off or unoptimized in some way. However not knowing how
> Drools
> > does its magic I'm currently inclined to think that Drools gets swamped
> > (10k in 300ms is a lot) and a few events get dropped so Drools can keep
> > operating
Re: [rules-users] Drools Fusion Dropping Events?
What you describe is definitely not "expected behaviour" - it's more like that race condition (or another one) already being in 5.3. Posting your code might be a good idea ;-) -W On 16/05/2013, Jason Barto wrote: > I've been working to load test Drools 5.3 to determine if its a fit for a > project. As part of the test I programmatically insert events as rapidly > as possible; an example, my earlier test inserted 10k events in about > 300ms. There is currently a single rule which reads the event and stores > the event's value into the only fact in Drools. I'm very happy to report, > and I'm sure it will be no surprise to anyone, that the engine processes > all the events in roughly 1 sec. However I have noticed that any large > number of events (~>1000) usually sees that a small number of events don't > get processed. I think after 10k events as many as 7 appear to have gone > unprocessed. If 100 events are inserted, rather than 10k, no events get > disregarded. Being new to Drools I can easily accept that my java code or > DRL code is off or unoptimized in some way. However not knowing how Drools > does its magic I'm currently inclined to think that Drools gets swamped > (10k in 300ms is a lot) and a few events get dropped so Drools can keep > operating. Is this a known or expected behavior from Drools? If not I am > happy to post my code, it is similar to the other code sets I've posted in > the last few days. I'm still new to Drools and trying to learn its behavior > so any insight or enlightenment is greatly appreciated. > > Sincerely, > Jason > ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Drools Fusion Dropping Events?
I've been working to load test Drools 5.3 to determine if its a fit for a project. As part of the test I programmatically insert events as rapidly as possible; an example, my earlier test inserted 10k events in about 300ms. There is currently a single rule which reads the event and stores the event's value into the only fact in Drools. I'm very happy to report, and I'm sure it will be no surprise to anyone, that the engine processes all the events in roughly 1 sec. However I have noticed that any large number of events (~>1000) usually sees that a small number of events don't get processed. I think after 10k events as many as 7 appear to have gone unprocessed. If 100 events are inserted, rather than 10k, no events get disregarded. Being new to Drools I can easily accept that my java code or DRL code is off or unoptimized in some way. However not knowing how Drools does its magic I'm currently inclined to think that Drools gets swamped (10k in 300ms is a lot) and a few events get dropped so Drools can keep operating. Is this a known or expected behavior from Drools? If not I am happy to post my code, it is similar to the other code sets I've posted in the last few days. I'm still new to Drools and trying to learn its behavior so any insight or enlightenment is greatly appreciated. Sincerely, Jason ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
