[rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Hi all,
We've been for long time now developing a complex event processing
system that ( simplified version ahead !! ) involves a set of rules that in
turn activates a set processes that fulfill human tasks and other kind of
tasks.. This all is running in a StatefulKnowledgeSession with JPA
persistence configured both for Drools and JBPM. We are running an stack of
Drools, JBPM, Drools Integration, Drools fussion, etc..
We've been able to persist Drools sessions JBPM processes in the same
Persistence Context ( not without pain :( ) using different JTA
implementations including ( but not limited to ) Bitronix Atomikos.
What we are observing here is what seems to be some kind of race
condition between Drools and JBPM when running the knowledge session when
JPAJTA persistence is configured. Very often, as soon as after 2-3
processes get created as rule's consequences are fired in response of
events inside the session we see how JBPM finds its instance nullified by
Drools when it tries to end a process and persist it.
We've been able to find where Drools decides to delete an instance of
the process ... at a given time Drools executes
JPAProcessInstanceManager.clearProcessInstances() [1] when it finalizes a
SingleSessionCommand wich in turn calls disconnect() for *all* the
local-stored processinstances ( wich gets populated with instances of
processes every time a process is started in the knowledge session ):
public void clearProcessInstances() {
for (ProcessInstance processInstance: new
ArrayListProcessInstance(processInstances.values())) {
((ProcessInstanceImpl) processInstance).disconnect();
}
}
So, Drools decides to disconnect all process instances in it's JPA
context without taking in account the state the process is in, and when an
processinstance that is not stopped gets removed then JBPM finds it's
NullPointerException...
We've modified the code to make Drools aware of the state of the
process before wiping it from the context ( no problem here, there will
be no leak as a running processinstance will be removed in future calls of
clearProcessInstances given the process is closed ). But unfortunatelly
this seems to resolve this problem, but lots of other problems ( wich seems
also race conditions arise : for instance: Drools closes connections to
the database and JBPM finds the connection closed,
So, we are really worried about using Drools JBPM in a persisted
environment. Maybe our asumptions are wrong... Is it possible to have an
scenario like ours given the current Drools JBPM integration status for a
persistent statefulKnowledge Session? Did anyone build a complex event
processing system like ours in a unaltered persistence environment such as
provided in Drools and JBPM by default?
Greets,
[1]
https://github.com/droolsjbpm/jbpm/blob/master/jbpm-persistence-jpa/src/main/java/org/jbpm/persistence/processinstance/JPAProcessInstanceManager.java
Alberto R. Galdo
arga...@gmail.co arga...@gmail.com
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Ok, so what is happening is that two or more threads are fighting for the
resources, causing that two or more transactions wants to be completed, but
just one can win. All the other must be retried. Are you seeing rolled back
exceptions in your stack trace?
My question to you is if you really need to handle everything in just one
big persistent session. I've solved similar issues in the past using more
than one session, for example one session to run business
processes(persistent) and one or more for receiving and reacting to events.
If you have an async way to communicate both sessions there should be no
problem.
Hope it helps!
Cheers
On Tue, May 29, 2012 at 7:30 AM, Alberto R. Galdo arga...@gmail.com wrote:
Hi all,
We've been for long time now developing a complex event processing
system that ( simplified version ahead !! ) involves a set of rules that in
turn activates a set processes that fulfill human tasks and other kind of
tasks.. This all is running in a StatefulKnowledgeSession with JPA
persistence configured both for Drools and JBPM. We are running an stack of
Drools, JBPM, Drools Integration, Drools fussion, etc..
We've been able to persist Drools sessions JBPM processes in the same
Persistence Context ( not without pain :( ) using different JTA
implementations including ( but not limited to ) Bitronix Atomikos.
What we are observing here is what seems to be some kind of race
condition between Drools and JBPM when running the knowledge session when
JPAJTA persistence is configured. Very often, as soon as after 2-3
processes get created as rule's consequences are fired in response of
events inside the session we see how JBPM finds its instance nullified by
Drools when it tries to end a process and persist it.
We've been able to find where Drools decides to delete an instance of
the process ... at a given time Drools executes
JPAProcessInstanceManager.clearProcessInstances() [1] when it finalizes a
SingleSessionCommand wich in turn calls disconnect() for *all* the
local-stored processinstances ( wich gets populated with instances of
processes every time a process is started in the knowledge session ):
public void clearProcessInstances() {
for (ProcessInstance processInstance: new
ArrayListProcessInstance(processInstances.values())) {
((ProcessInstanceImpl) processInstance).disconnect();
}
}
So, Drools decides to disconnect all process instances in it's JPA
context without taking in account the state the process is in, and when an
processinstance that is not stopped gets removed then JBPM finds it's
NullPointerException...
We've modified the code to make Drools aware of the state of the
process before wiping it from the context ( no problem here, there will
be no leak as a running processinstance will be removed in future calls of
clearProcessInstances given the process is closed ). But unfortunatelly
this seems to resolve this problem, but lots of other problems ( wich seems
also race conditions arise : for instance: Drools closes connections to
the database and JBPM finds the connection closed,
So, we are really worried about using Drools JBPM in a persisted
environment. Maybe our asumptions are wrong... Is it possible to have an
scenario like ours given the current Drools JBPM integration status for a
persistent statefulKnowledge Session? Did anyone build a complex event
processing system like ours in a unaltered persistence environment such as
provided in Drools and JBPM by default?
Greets,
[1]
https://github.com/droolsjbpm/jbpm/blob/master/jbpm-persistence-jpa/src/main/java/org/jbpm/persistence/processinstance/JPAProcessInstanceManager.java
Alberto R. Galdo
arga...@gmail.co arga...@gmail.com
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
- MyJourney @ http://salaboy.wordpress.com
- Co-Founder @ http://www.jugargentina.org
- Co-Founder @ http://www.jbug.com.ar
- Salatino Salaboy Mauricio -
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Thanks Mauricio for your kind response, and kick, as usual ;-)
Sure, we see lots of rolled back transactions ... Everywhere!
Let me see if I'm getting this right, What you mean is that there is no
way for the JPA persistence of Drools JBPM to coexist in the same
knowledge session because of race conditions between them competing for
their shared resources? I'm afraid so.. :(
If it is so, Are there any plans for the integration of a shared JPA
persistence management of both Drools JBPM, wich may be the solution for
this problem? ... Will we be better thinking to change our architecture
ASAP?
Greets,
Alberto R. Galdo
arga...@gmail.com
On Tue, May 29, 2012 at 1:54 PM, Mauricio Salatino sala...@gmail.comwrote:
Ok, so what is happening is that two or more threads are fighting for the
resources, causing that two or more transactions wants to be completed, but
just one can win. All the other must be retried. Are you seeing rolled back
exceptions in your stack trace?
My question to you is if you really need to handle everything in just one
big persistent session. I've solved similar issues in the past using more
than one session, for example one session to run business
processes(persistent) and one or more for receiving and reacting to events.
If you have an async way to communicate both sessions there should be no
problem.
Hope it helps!
Cheers
On Tue, May 29, 2012 at 7:30 AM, Alberto R. Galdo arga...@gmail.comwrote:
Hi all,
We've been for long time now developing a complex event processing
system that ( simplified version ahead !! ) involves a set of rules that in
turn activates a set processes that fulfill human tasks and other kind of
tasks.. This all is running in a StatefulKnowledgeSession with JPA
persistence configured both for Drools and JBPM. We are running an stack of
Drools, JBPM, Drools Integration, Drools fussion, etc..
We've been able to persist Drools sessions JBPM processes in the
same Persistence Context ( not without pain :( ) using different JTA
implementations including ( but not limited to ) Bitronix Atomikos.
What we are observing here is what seems to be some kind of race
condition between Drools and JBPM when running the knowledge session when
JPAJTA persistence is configured. Very often, as soon as after 2-3
processes get created as rule's consequences are fired in response of
events inside the session we see how JBPM finds its instance nullified by
Drools when it tries to end a process and persist it.
We've been able to find where Drools decides to delete an instance of
the process ... at a given time Drools executes
JPAProcessInstanceManager.clearProcessInstances() [1] when it finalizes a
SingleSessionCommand wich in turn calls disconnect() for *all* the
local-stored processinstances ( wich gets populated with instances of
processes every time a process is started in the knowledge session ):
public void clearProcessInstances() {
for (ProcessInstance processInstance: new
ArrayListProcessInstance(processInstances.values())) {
((ProcessInstanceImpl) processInstance).disconnect();
}
}
So, Drools decides to disconnect all process instances in it's JPA
context without taking in account the state the process is in, and when an
processinstance that is not stopped gets removed then JBPM finds it's
NullPointerException...
We've modified the code to make Drools aware of the state of the
process before wiping it from the context ( no problem here, there will
be no leak as a running processinstance will be removed in future calls of
clearProcessInstances given the process is closed ). But unfortunatelly
this seems to resolve this problem, but lots of other problems ( wich seems
also race conditions arise : for instance: Drools closes connections to
the database and JBPM finds the connection closed,
So, we are really worried about using Drools JBPM in a persisted
environment. Maybe our asumptions are wrong... Is it possible to have an
scenario like ours given the current Drools JBPM integration status for a
persistent statefulKnowledge Session? Did anyone build a complex event
processing system like ours in a unaltered persistence environment such as
provided in Drools and JBPM by default?
Greets,
[1]
https://github.com/droolsjbpm/jbpm/blob/master/jbpm-persistence-jpa/src/main/java/org/jbpm/persistence/processinstance/JPAProcessInstanceManager.java
Alberto R. Galdo
arga...@gmail.co arga...@gmail.com
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
- MyJourney @ http://salaboy.wordpress.com
- Co-Founder @ http://www.jugargentina.org
- Co-Founder @ http://www.jbug.com.ar
- Salatino Salaboy Mauricio -
___
rules-users mailing list
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
I'm in no way saying this:
Let me see if I'm getting this right, What you mean is that there is no
way for the JPA persistence of Drools JBPM to coexist in the same
knowledge session because of race conditions between them competing for
their shared resources? I'm afraid so.. :(
They can coexist. As you can see they are working together. If you take a
look at the underlying layers you will see that each operation that you run
against the session is wrapped in a transaction. That leads to your
architecture, If you have a single entry point for your session everything
will work correctly. You can achieve this by receiving all the operations
in a queue and then execute one after the other.
If you have multiple threads interacting with the session, you can
implement a retrying mechanism if the transaction gets rolled back and as
soon as the transaction win the right to commit it will work. It really
depends on the problem that you are trying to solve.
Cheers
On Tue, May 29, 2012 at 10:42 AM, Alberto R. Galdo arga...@gmail.comwrote:
Thanks Mauricio for your kind response, and kick, as usual ;-)
Sure, we see lots of rolled back transactions ... Everywhere!
Let me see if I'm getting this right, What you mean is that there is no
way for the JPA persistence of Drools JBPM to coexist in the same
knowledge session because of race conditions between them competing for
their shared resources? I'm afraid so.. :(
If it is so, Are there any plans for the integration of a shared JPA
persistence management of both Drools JBPM, wich may be the solution for
this problem? ... Will we be better thinking to change our architecture
ASAP?
Greets,
Alberto R. Galdo
arga...@gmail.com
On Tue, May 29, 2012 at 1:54 PM, Mauricio Salatino sala...@gmail.comwrote:
Ok, so what is happening is that two or more threads are fighting for the
resources, causing that two or more transactions wants to be completed, but
just one can win. All the other must be retried. Are you seeing rolled back
exceptions in your stack trace?
My question to you is if you really need to handle everything in just one
big persistent session. I've solved similar issues in the past using more
than one session, for example one session to run business
processes(persistent) and one or more for receiving and reacting to events.
If you have an async way to communicate both sessions there should be no
problem.
Hope it helps!
Cheers
On Tue, May 29, 2012 at 7:30 AM, Alberto R. Galdo arga...@gmail.comwrote:
Hi all,
We've been for long time now developing a complex event processing
system that ( simplified version ahead !! ) involves a set of rules that in
turn activates a set processes that fulfill human tasks and other kind of
tasks.. This all is running in a StatefulKnowledgeSession with JPA
persistence configured both for Drools and JBPM. We are running an stack of
Drools, JBPM, Drools Integration, Drools fussion, etc..
We've been able to persist Drools sessions JBPM processes in the
same Persistence Context ( not without pain :( ) using different JTA
implementations including ( but not limited to ) Bitronix Atomikos.
What we are observing here is what seems to be some kind of race
condition between Drools and JBPM when running the knowledge session when
JPAJTA persistence is configured. Very often, as soon as after 2-3
processes get created as rule's consequences are fired in response of
events inside the session we see how JBPM finds its instance nullified by
Drools when it tries to end a process and persist it.
We've been able to find where Drools decides to delete an instance of
the process ... at a given time Drools executes
JPAProcessInstanceManager.clearProcessInstances() [1] when it finalizes a
SingleSessionCommand wich in turn calls disconnect() for *all* the
local-stored processinstances ( wich gets populated with instances of
processes every time a process is started in the knowledge session ):
public void clearProcessInstances() {
for (ProcessInstance processInstance: new
ArrayListProcessInstance(processInstances.values())) {
((ProcessInstanceImpl) processInstance).disconnect();
}
}
So, Drools decides to disconnect all process instances in it's JPA
context without taking in account the state the process is in, and when an
processinstance that is not stopped gets removed then JBPM finds it's
NullPointerException...
We've modified the code to make Drools aware of the state of the
process before wiping it from the context ( no problem here, there will
be no leak as a running processinstance will be removed in future calls of
clearProcessInstances given the process is closed ). But unfortunatelly
this seems to resolve this problem, but lots of other problems ( wich seems
also race conditions arise : for instance: Drools closes connections to
the database and JBPM finds the connection closed,
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Your answer is confusing me, I think I'm not getting it right. It seems that we you're saying that will have to modify JBPM or Drools behaviour to get our architecture done ... but I don't think we are doing nothing special than what Drools JBPM are able to do in a non-persisting enviroment ( at least without JBPM persisted ). Maybe deeping a bit more the architecture will lead to a better understanding of the problem: We have a Drools StatefulKnowledgeSession wich is populated with packages of rules and different BPMN 2.0 process definitions. That session in Drools is persisted using JPA and our JTA provider. Our solution is a CEP that uses events that get into the session using Drools Fusion. There, several rules fire and then several consequences start processes wich contain both Tasks and HumanTasks. Those rule activations are already in a queue, the Agenda. Given that JBPM acts as the processmanager provider for Drools, our processes get started in JBPM, and it is persisted using it's own JPA manager. Just rules, that start processes in response to events in a fireUntilHalt loop in a persistent enviroment, both for Drools JBPM. Then some processes interact with the knowledge session using BussinessTask nodes( which make other rules fire and maybe launch a different set of processes ) and other simply get to the Stop node. I see nothing wrong here, in fact, taking the persistence appart from the equation, everything runs as beautifuly as expected. We've tested that. I'm just trying to undersand the problem, I'm pretty 90% sure I'm wrong, but what I think is happening here is that Drools isn't aware that JBPM may be using the same resources and vice-versa, Isn't it? Shouldn't the solution be to make Drools JBPM aware of each other in what is related to the syncronization of the transaction management to avoid race conditions? Alberto R. Galdo arga...@gmail.com On Tue, May 29, 2012 at 3:58 PM, Mauricio Salatino sala...@gmail.comwrote: I'm in no way saying this: Let me see if I'm getting this right, What you mean is that there is no way for the JPA persistence of Drools JBPM to coexist in the same knowledge session because of race conditions between them competing for their shared resources? I'm afraid so.. :( They can coexist. As you can see they are working together. If you take a look at the underlying layers you will see that each operation that you run against the session is wrapped in a transaction. That leads to your architecture, If you have a single entry point for your session everything will work correctly. You can achieve this by receiving all the operations in a queue and then execute one after the other. If you have multiple threads interacting with the session, you can implement a retrying mechanism if the transaction gets rolled back and as soon as the transaction win the right to commit it will work. It really depends on the problem that you are trying to solve. Cheers On Tue, May 29, 2012 at 10:42 AM, Alberto R. Galdo arga...@gmail.comwrote: Thanks Mauricio for your kind response, and kick, as usual ;-) Sure, we see lots of rolled back transactions ... Everywhere! Let me see if I'm getting this right, What you mean is that there is no way for the JPA persistence of Drools JBPM to coexist in the same knowledge session because of race conditions between them competing for their shared resources? I'm afraid so.. :( If it is so, Are there any plans for the integration of a shared JPA persistence management of both Drools JBPM, wich may be the solution for this problem? ... Will we be better thinking to change our architecture ASAP? Greets, Alberto R. Galdo arga...@gmail.com On Tue, May 29, 2012 at 1:54 PM, Mauricio Salatino sala...@gmail.comwrote: Ok, so what is happening is that two or more threads are fighting for the resources, causing that two or more transactions wants to be completed, but just one can win. All the other must be retried. Are you seeing rolled back exceptions in your stack trace? My question to you is if you really need to handle everything in just one big persistent session. I've solved similar issues in the past using more than one session, for example one session to run business processes(persistent) and one or more for receiving and reacting to events. If you have an async way to communicate both sessions there should be no problem. Hope it helps! Cheers On Tue, May 29, 2012 at 7:30 AM, Alberto R. Galdo arga...@gmail.comwrote: Hi all, We've been for long time now developing a complex event processing system that ( simplified version ahead !! ) involves a set of rules that in turn activates a set processes that fulfill human tasks and other kind of tasks.. This all is running in a StatefulKnowledgeSession with JPA persistence configured both for Drools and JBPM. We are running an stack of Drools, JBPM, Drools Integration, Drools fussion, etc..
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Ok, so probably I'm not understanding your scenario, but are you using the JPAKnowledgeService to create persistent sessions? If you use that everything is aware of everything :) Drools and jBPM5 will run using the same persistence resources. For each interaction: startProcess, insert, update, fireAllRules a transaction will be created. If you are starting a process and at the same time inserting a Fusion event you will be generating two transaction that will fight for the resources (the same resources because you have only one session). Obviously if you take the persistence mechanisms outside of the discussion everything will work perfectly fine, because you don't have any transactional resource to fight for :) Hope it helps. Cheers On Tue, May 29, 2012 at 11:36 AM, Alberto R. Galdo arga...@gmail.comwrote: Your answer is confusing me, I think I'm not getting it right. It seems that we you're saying that will have to modify JBPM or Drools behaviour to get our architecture done ... but I don't think we are doing nothing special than what Drools JBPM are able to do in a non-persisting enviroment ( at least without JBPM persisted ). Maybe deeping a bit more the architecture will lead to a better understanding of the problem: We have a Drools StatefulKnowledgeSession wich is populated with packages of rules and different BPMN 2.0 process definitions. That session in Drools is persisted using JPA and our JTA provider. Our solution is a CEP that uses events that get into the session using Drools Fusion. There, several rules fire and then several consequences start processes wich contain both Tasks and HumanTasks. Those rule activations are already in a queue, the Agenda. Given that JBPM acts as the processmanager provider for Drools, our processes get started in JBPM, and it is persisted using it's own JPA manager. Just rules, that start processes in response to events in a fireUntilHalt loop in a persistent enviroment, both for Drools JBPM. Then some processes interact with the knowledge session using BussinessTask nodes( which make other rules fire and maybe launch a different set of processes ) and other simply get to the Stop node. I see nothing wrong here, in fact, taking the persistence appart from the equation, everything runs as beautifuly as expected. We've tested that. I'm just trying to undersand the problem, I'm pretty 90% sure I'm wrong, but what I think is happening here is that Drools isn't aware that JBPM may be using the same resources and vice-versa, Isn't it? Shouldn't the solution be to make Drools JBPM aware of each other in what is related to the syncronization of the transaction management to avoid race conditions? Alberto R. Galdo arga...@gmail.com On Tue, May 29, 2012 at 3:58 PM, Mauricio Salatino sala...@gmail.comwrote: I'm in no way saying this: Let me see if I'm getting this right, What you mean is that there is no way for the JPA persistence of Drools JBPM to coexist in the same knowledge session because of race conditions between them competing for their shared resources? I'm afraid so.. :( They can coexist. As you can see they are working together. If you take a look at the underlying layers you will see that each operation that you run against the session is wrapped in a transaction. That leads to your architecture, If you have a single entry point for your session everything will work correctly. You can achieve this by receiving all the operations in a queue and then execute one after the other. If you have multiple threads interacting with the session, you can implement a retrying mechanism if the transaction gets rolled back and as soon as the transaction win the right to commit it will work. It really depends on the problem that you are trying to solve. Cheers On Tue, May 29, 2012 at 10:42 AM, Alberto R. Galdo arga...@gmail.comwrote: Thanks Mauricio for your kind response, and kick, as usual ;-) Sure, we see lots of rolled back transactions ... Everywhere! Let me see if I'm getting this right, What you mean is that there is no way for the JPA persistence of Drools JBPM to coexist in the same knowledge session because of race conditions between them competing for their shared resources? I'm afraid so.. :( If it is so, Are there any plans for the integration of a shared JPA persistence management of both Drools JBPM, wich may be the solution for this problem? ... Will we be better thinking to change our architecture ASAP? Greets, Alberto R. Galdo arga...@gmail.com On Tue, May 29, 2012 at 1:54 PM, Mauricio Salatino sala...@gmail.comwrote: Ok, so what is happening is that two or more threads are fighting for the resources, causing that two or more transactions wants to be completed, but just one can win. All the other must be retried. Are you seeing rolled back exceptions in your stack trace? My question to you is if you really need to handle
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Sure!. I didn't mention that we're running in a OSGi environment and
we're getting the session from JPAKnowledgeService like that:
EntityManagerFactory emf =
Persistence.createEntityManagerFactory(com.package);
Environment env = KnowledgeBaseFactory.newEnvironment();
env.set(EnvironmentName.ENTITY_MANAGER_FACTORY, emf);
env.set(EnvironmentName.TRANSACTION_MANAGER,(TransactionManager)
bundleContext.getService(this.tmServiceRef));
env.set(EnvironmentName.TRANSACTION, (UserTransaction)
bundleContext.getService(this.userTransactionServiceRef));
env.set(EnvironmentName.GLOBALS, new MapGlobalResolver());
env.set(EnvironmentName.OBJECT_MARSHALLING_STRATEGIES, new
ObjectMarshallingStrategy[] {
MarshallerFactory.newSerializeMarshallingStrategy() });
final StatefulKnowledgeSession ksession =
JPAKnowledgeService.newStatefulKnowledgeSession(kbase, sessionConfig, env);
We are just following the manual ( and some acquired knowledge through
the days ... ) I understand that inserting an event and starting a
process at the same time would lead to the current situation, but Isn't
this the reason why we are using drools with drools fusion + jbpm to build
a CEP?
We chose Drools and a statefulsession in a fireUntilHalt loop just
because we would be able to have our rules and processes reacting to events
that enter the system at their own pace ... and found ourselves in this
nightmare when trying to make the system fault-tolerant through
persistence... :(
Do you think we should change our fault-tolerance strategy by
reconstructing the knowledge session, process and tasks state by our own
means leaving Drools and JBPM JPA persistence appart?
Alberto R. Galdo
arga...@gmail.com
On Tue, May 29, 2012 at 4:43 PM, Mauricio Salatino sala...@gmail.comwrote:
Ok, so probably I'm not understanding your scenario, but are you using the
JPAKnowledgeService to create persistent sessions?
If you use that everything is aware of everything :) Drools and jBPM5 will
run using the same persistence resources. For each interaction:
startProcess, insert, update, fireAllRules a transaction will be created.
If you are starting a process and at the same time inserting a Fusion event
you will be generating two transaction that will fight for the resources
(the same resources because you have only one session).
Obviously if you take the persistence mechanisms outside of the discussion
everything will work perfectly fine, because you don't have any
transactional resource to fight for :)
Hope it helps.
Cheers
On Tue, May 29, 2012 at 11:36 AM, Alberto R. Galdo arga...@gmail.comwrote:
Your answer is confusing me, I think I'm not getting it right.
It seems that we you're saying that will have to modify JBPM or Drools
behaviour to get our architecture done ... but I don't think we are doing
nothing special than what Drools JBPM are able to do in a non-persisting
enviroment ( at least without JBPM persisted ).
Maybe deeping a bit more the architecture will lead to a better
understanding of the problem:
We have a Drools StatefulKnowledgeSession wich is populated with
packages of rules and different BPMN 2.0 process definitions. That session
in Drools is persisted using JPA and our JTA provider. Our solution is a
CEP that uses events that get into the session using Drools Fusion. There,
several rules fire and then several consequences start processes wich
contain both Tasks and HumanTasks. Those rule activations are already in a
queue, the Agenda. Given that JBPM acts as the processmanager provider for
Drools, our processes get started in JBPM, and it is persisted using it's
own JPA manager. Just rules, that start processes in response to events in
a fireUntilHalt loop in a persistent enviroment, both for Drools JBPM.
Then some processes interact with the knowledge session using BussinessTask
nodes( which make other rules fire and maybe launch a different set of
processes ) and other simply get to the Stop node. I see nothing wrong
here, in fact, taking the persistence appart from the equation, everything
runs as beautifuly as expected. We've tested that.
I'm just trying to undersand the problem, I'm pretty 90% sure I'm
wrong, but what I think is happening here is that Drools isn't aware that
JBPM may be using the same resources and vice-versa, Isn't it?
Shouldn't the solution be to make Drools JBPM aware of each other in
what is related to the syncronization of the transaction management to
avoid race conditions?
Alberto R. Galdo
arga...@gmail.com
On Tue, May 29, 2012 at 3:58 PM, Mauricio Salatino sala...@gmail.comwrote:
I'm in no way saying this:
Let me see if I'm getting this right, What you mean is that there is
no way for the JPA persistence of Drools JBPM to coexist in the same
knowledge session because of race conditions between them competing for
their shared resources? I'm afraid so.. :(
They can coexist. As you can see they are
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Comments inline,
On Tue, May 29, 2012 at 12:24 PM, Alberto R. Galdo arga...@gmail.comwrote:
Sure!. I didn't mention that we're running in a OSGi environment and
we're getting the session from JPAKnowledgeService like that:
EntityManagerFactory emf =
Persistence.createEntityManagerFactory(com.package);
Environment env = KnowledgeBaseFactory.newEnvironment();
env.set(EnvironmentName.ENTITY_MANAGER_FACTORY, emf);
env.set(EnvironmentName.TRANSACTION_MANAGER,(TransactionManager)
bundleContext.getService(this.tmServiceRef));
env.set(EnvironmentName.TRANSACTION, (UserTransaction)
bundleContext.getService(this.userTransactionServiceRef));
env.set(EnvironmentName.GLOBALS, new MapGlobalResolver());
env.set(EnvironmentName.OBJECT_MARSHALLING_STRATEGIES, new
ObjectMarshallingStrategy[] {
MarshallerFactory.newSerializeMarshallingStrategy() });
final StatefulKnowledgeSession ksession =
JPAKnowledgeService.newStatefulKnowledgeSession(kbase, sessionConfig, env);
We are just following the manual ( and some acquired knowledge through
the days ... ) I understand that inserting an event and starting a
process at the same time would lead to the current situation, but Isn't
this the reason why we are using drools with drools fusion + jbpm to build
a CEP?
I'm not sure about your reasons :)
We chose Drools and a statefulsession in a fireUntilHalt loop just
because we would be able to have our rules and processes reacting to events
that enter the system at their own pace ... and found ourselves in this
nightmare when trying to make the system fault-tolerant through
persistence... :(
FireUntilHalt and Persistence will cause conflicts, because
fireUntilHalts by default creates a different thread == race conditions.
Do you think we should change our fault-tolerance strategy by
reconstructing the knowledge session, process and tasks state by our own
means leaving Drools and JBPM JPA persistence appart?
A common problem is to think that a session will do everything for you.
I'm not saying that you need to separate your processes from your rules,
I'm just saying that probably decoupling responsibilities will help you to
tackle down some of the problems that you are having. If you have in memory
processes and fusion entry points, you can keep those together and in a
separate persistence session handle all the human task interactions and
long running processes. If you are getting events in real time and you want
to process them, you cannot expect to persist all the session state and
have no performance impacts.
Alberto R. Galdo
arga...@gmail.com
Cheers
On Tue, May 29, 2012 at 4:43 PM, Mauricio Salatino sala...@gmail.comwrote:
Ok, so probably I'm not understanding your scenario, but are you using
the JPAKnowledgeService to create persistent sessions?
If you use that everything is aware of everything :) Drools and jBPM5
will run using the same persistence resources. For each interaction:
startProcess, insert, update, fireAllRules a transaction will be created.
If you are starting a process and at the same time inserting a Fusion event
you will be generating two transaction that will fight for the resources
(the same resources because you have only one session).
Obviously if you take the persistence mechanisms outside of the
discussion everything will work perfectly fine, because you don't have any
transactional resource to fight for :)
Hope it helps.
Cheers
On Tue, May 29, 2012 at 11:36 AM, Alberto R. Galdo arga...@gmail.comwrote:
Your answer is confusing me, I think I'm not getting it right.
It seems that we you're saying that will have to modify JBPM or
Drools behaviour to get our architecture done ... but I don't think we are
doing nothing special than what Drools JBPM are able to do in a
non-persisting enviroment ( at least without JBPM persisted ).
Maybe deeping a bit more the architecture will lead to a better
understanding of the problem:
We have a Drools StatefulKnowledgeSession wich is populated with
packages of rules and different BPMN 2.0 process definitions. That session
in Drools is persisted using JPA and our JTA provider. Our solution is a
CEP that uses events that get into the session using Drools Fusion. There,
several rules fire and then several consequences start processes wich
contain both Tasks and HumanTasks. Those rule activations are already in a
queue, the Agenda. Given that JBPM acts as the processmanager provider for
Drools, our processes get started in JBPM, and it is persisted using it's
own JPA manager. Just rules, that start processes in response to events in
a fireUntilHalt loop in a persistent enviroment, both for Drools JBPM.
Then some processes interact with the knowledge session using BussinessTask
nodes( which make other rules fire and maybe launch a different set of
processes ) and other simply get to the Stop node. I see nothing wrong
here, in fact, taking the
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
Thank you very much Mauricio. You've been of great help. Now is time for us
to cook all this valuable information and go for a solution. xD
Alberto R. Galdo
arga...@gmail.com
On Tue, May 29, 2012 at 5:59 PM, Mauricio Salatino sala...@gmail.comwrote:
Comments inline,
On Tue, May 29, 2012 at 12:24 PM, Alberto R. Galdo arga...@gmail.comwrote:
Sure!. I didn't mention that we're running in a OSGi environment and
we're getting the session from JPAKnowledgeService like that:
EntityManagerFactory emf =
Persistence.createEntityManagerFactory(com.package);
Environment env = KnowledgeBaseFactory.newEnvironment();
env.set(EnvironmentName.ENTITY_MANAGER_FACTORY, emf);
env.set(EnvironmentName.TRANSACTION_MANAGER,(TransactionManager)
bundleContext.getService(this.tmServiceRef));
env.set(EnvironmentName.TRANSACTION, (UserTransaction)
bundleContext.getService(this.userTransactionServiceRef));
env.set(EnvironmentName.GLOBALS, new MapGlobalResolver());
env.set(EnvironmentName.OBJECT_MARSHALLING_STRATEGIES, new
ObjectMarshallingStrategy[] {
MarshallerFactory.newSerializeMarshallingStrategy() });
final StatefulKnowledgeSession ksession =
JPAKnowledgeService.newStatefulKnowledgeSession(kbase, sessionConfig, env);
We are just following the manual ( and some acquired knowledge through
the days ... ) I understand that inserting an event and starting a
process at the same time would lead to the current situation, but Isn't
this the reason why we are using drools with drools fusion + jbpm to build
a CEP?
I'm not sure about your reasons :)
We chose Drools and a statefulsession in a fireUntilHalt loop just
because we would be able to have our rules and processes reacting to events
that enter the system at their own pace ... and found ourselves in this
nightmare when trying to make the system fault-tolerant through
persistence... :(
FireUntilHalt and Persistence will cause conflicts, because
fireUntilHalts by default creates a different thread == race conditions.
Do you think we should change our fault-tolerance strategy by
reconstructing the knowledge session, process and tasks state by our own
means leaving Drools and JBPM JPA persistence appart?
A common problem is to think that a session will do everything for you.
I'm not saying that you need to separate your processes from your rules,
I'm just saying that probably decoupling responsibilities will help you to
tackle down some of the problems that you are having. If you have in memory
processes and fusion entry points, you can keep those together and in a
separate persistence session handle all the human task interactions and
long running processes. If you are getting events in real time and you want
to process them, you cannot expect to persist all the session state and
have no performance impacts.
Alberto R. Galdo
arga...@gmail.com
Cheers
On Tue, May 29, 2012 at 4:43 PM, Mauricio Salatino sala...@gmail.comwrote:
Ok, so probably I'm not understanding your scenario, but are you using
the JPAKnowledgeService to create persistent sessions?
If you use that everything is aware of everything :) Drools and jBPM5
will run using the same persistence resources. For each interaction:
startProcess, insert, update, fireAllRules a transaction will be created.
If you are starting a process and at the same time inserting a Fusion event
you will be generating two transaction that will fight for the resources
(the same resources because you have only one session).
Obviously if you take the persistence mechanisms outside of the
discussion everything will work perfectly fine, because you don't have any
transactional resource to fight for :)
Hope it helps.
Cheers
On Tue, May 29, 2012 at 11:36 AM, Alberto R. Galdo arga...@gmail.comwrote:
Your answer is confusing me, I think I'm not getting it right.
It seems that we you're saying that will have to modify JBPM or
Drools behaviour to get our architecture done ... but I don't think we are
doing nothing special than what Drools JBPM are able to do in a
non-persisting enviroment ( at least without JBPM persisted ).
Maybe deeping a bit more the architecture will lead to a better
understanding of the problem:
We have a Drools StatefulKnowledgeSession wich is populated with
packages of rules and different BPMN 2.0 process definitions. That session
in Drools is persisted using JPA and our JTA provider. Our solution is a
CEP that uses events that get into the session using Drools Fusion. There,
several rules fire and then several consequences start processes wich
contain both Tasks and HumanTasks. Those rule activations are already in a
queue, the Agenda. Given that JBPM acts as the processmanager provider for
Drools, our processes get started in JBPM, and it is persisted using it's
own JPA manager. Just rules, that start processes in response to events in
a fireUntilHalt loop in a persistent enviroment, both for
Re: [rules-users] Possibly race condition in a persisted Drools + JBPM during a StatefulKnowledgeSession
:)
Keep your mind open, because there is no single solution for all the
problems.
On Tue, May 29, 2012 at 1:15 PM, Alberto R. Galdo arga...@gmail.com wrote:
Thank you very much Mauricio. You've been of great help. Now is time for
us to cook all this valuable information and go for a solution. xD
Alberto R. Galdo
arga...@gmail.com
On Tue, May 29, 2012 at 5:59 PM, Mauricio Salatino sala...@gmail.comwrote:
Comments inline,
On Tue, May 29, 2012 at 12:24 PM, Alberto R. Galdo arga...@gmail.comwrote:
Sure!. I didn't mention that we're running in a OSGi environment and
we're getting the session from JPAKnowledgeService like that:
EntityManagerFactory emf =
Persistence.createEntityManagerFactory(com.package);
Environment env = KnowledgeBaseFactory.newEnvironment();
env.set(EnvironmentName.ENTITY_MANAGER_FACTORY, emf);
env.set(EnvironmentName.TRANSACTION_MANAGER,(TransactionManager)
bundleContext.getService(this.tmServiceRef));
env.set(EnvironmentName.TRANSACTION, (UserTransaction)
bundleContext.getService(this.userTransactionServiceRef));
env.set(EnvironmentName.GLOBALS, new MapGlobalResolver());
env.set(EnvironmentName.OBJECT_MARSHALLING_STRATEGIES, new
ObjectMarshallingStrategy[] {
MarshallerFactory.newSerializeMarshallingStrategy() });
final StatefulKnowledgeSession ksession =
JPAKnowledgeService.newStatefulKnowledgeSession(kbase, sessionConfig, env);
We are just following the manual ( and some acquired knowledge
through the days ... ) I understand that inserting an event and
starting a process at the same time would lead to the current situation,
but Isn't this the reason why we are using drools with drools fusion + jbpm
to build a CEP?
I'm not sure about your reasons :)
We chose Drools and a statefulsession in a fireUntilHalt loop just
because we would be able to have our rules and processes reacting to events
that enter the system at their own pace ... and found ourselves in this
nightmare when trying to make the system fault-tolerant through
persistence... :(
FireUntilHalt and Persistence will cause conflicts, because
fireUntilHalts by default creates a different thread == race conditions.
Do you think we should change our fault-tolerance strategy by
reconstructing the knowledge session, process and tasks state by our own
means leaving Drools and JBPM JPA persistence appart?
A common problem is to think that a session will do everything for you.
I'm not saying that you need to separate your processes from your rules,
I'm just saying that probably decoupling responsibilities will help you to
tackle down some of the problems that you are having. If you have in memory
processes and fusion entry points, you can keep those together and in a
separate persistence session handle all the human task interactions and
long running processes. If you are getting events in real time and you want
to process them, you cannot expect to persist all the session state and
have no performance impacts.
Alberto R. Galdo
arga...@gmail.com
Cheers
On Tue, May 29, 2012 at 4:43 PM, Mauricio Salatino sala...@gmail.comwrote:
Ok, so probably I'm not understanding your scenario, but are you using
the JPAKnowledgeService to create persistent sessions?
If you use that everything is aware of everything :) Drools and jBPM5
will run using the same persistence resources. For each interaction:
startProcess, insert, update, fireAllRules a transaction will be created.
If you are starting a process and at the same time inserting a Fusion event
you will be generating two transaction that will fight for the resources
(the same resources because you have only one session).
Obviously if you take the persistence mechanisms outside of the
discussion everything will work perfectly fine, because you don't have any
transactional resource to fight for :)
Hope it helps.
Cheers
On Tue, May 29, 2012 at 11:36 AM, Alberto R. Galdo
arga...@gmail.comwrote:
Your answer is confusing me, I think I'm not getting it right.
It seems that we you're saying that will have to modify JBPM or
Drools behaviour to get our architecture done ... but I don't think we are
doing nothing special than what Drools JBPM are able to do in a
non-persisting enviroment ( at least without JBPM persisted ).
Maybe deeping a bit more the architecture will lead to a better
understanding of the problem:
We have a Drools StatefulKnowledgeSession wich is populated with
packages of rules and different BPMN 2.0 process definitions. That session
in Drools is persisted using JPA and our JTA provider. Our solution is a
CEP that uses events that get into the session using Drools Fusion. There,
several rules fire and then several consequences start processes wich
contain both Tasks and HumanTasks. Those rule activations are already in a
queue, the Agenda. Given that JBPM acts as the processmanager provider for
Drools, our processes get started in JBPM,
