Smarter Multi-threading (was: Discussion: Entity Engine Redesign)

2014-08-27 Thread Adrian Crum
On 8/27/2014 11:17 AM, Scott Gray wrote: I don't have a problem with multi-threaded transactions, I'm not sure under what situations I would use it but I'm not against it. I have a better understanding now of why you don't want to use ThreadLocal, thanks. In our current design, an incoming r

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Adrian Crum
Yes, I am referring to separate transactions using suspend/resume. I thought I mentioned that. I apologize for the confusion. Behavioral Changes -- Current (simplified for clarity): Transaction suspendedTransaction = null; try { suspendedTransaction = TransactionUtil.suspend(

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Scott Gray
I don't have a problem with multi-threaded transactions, I'm not sure under what situations I would use it but I'm not against it. I have a better understanding now of why you don't want to use ThreadLocal, thanks. Regarding your code simplification, I'm not sure I understand the concept your

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Jacques Le Roux
Thank you Adrian, I truly appreciate your summary and your proposition. I'm back after a painful week and I will hopefully cope with the hectic commits and important conversations which happened this week. At least I can already say I agree most of the time ThreadLocal variables are abused. Ja

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Adrian Crum
Fair enough. Perhaps I feel like I get immediate push-back on my ideas, and I feel like no one is taking me seriously. Those feelings are my problem and I will deal with them. Meanwhile, I gave a very detailed reply in another post. Adrian Crum Sandglass Software www.sandglass-software.com On

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Scott Gray
Okay sure but you can't see how this thread has been a bit confusing? You've mixed two features you'd like to implement, multi-threaded transactions and an API rewrite, with an unrelated caching problem. You're also talking about problems with when commits occur that I haven't yet been able t

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Adrian Crum
Okay, let's start over from the beginning. This will be lengthy... Around 2010 I was trying various data modelling scenarios and the "ant load-demo" task was taking to long to execute, slowing down progress. So, I started experimenting with a multi-threaded data loader. I was able to get the a

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Adrian Crum
>> On 8/27/2014 9:30 AM, Scott Gray wrote: >>> You're yet to describe the problem with using ThreadLocal variables. > On 25/08/2014, at 10:51 am, Adrian Crum wrote: >> A Delegator instance always references the transaction it is running in. >> >> The advantage

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Scott Gray
Okay whatever, thanks for wasting my time with this thread. On 27/08/2014, at 9:39 am, Adrian Crum wrote: > I have, but no one is paying attention. Perhaps if I worked for Hotwax, I > could get someone to pay attention. > > I'm done. > > Adrian Crum > Sandglass Software > www.sandglass-softw

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Adrian Crum
I have, but no one is paying attention. Perhaps if I worked for Hotwax, I could get someone to pay attention. I'm done. Adrian Crum Sandglass Software www.sandglass-software.com On 8/27/2014 9:30 AM, Scott Gray wrote: You're yet to describe the problem with using ThreadLocal variables. If y

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Scott Gray
You're yet to describe the problem with using ThreadLocal variables. >>> If you read the Jira issue, I point out another problem with this clunky >>> implementation - calling "commit" doesn't really commit the transaction. >>> That is why we end up with invalid data in the entity cache - because

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Jacopo Cappellato
On Aug 27, 2014, at 9:36 AM, Adrian Crum wrote: > We're not using EJBs [...] I just quoted the full text: the fact that we don't use EJB is not relevant because that descriptions applies to OFBiz pretty well. Jacopo

Re: Discussion: Entity Engine Redesign

2014-08-27 Thread Adrian Crum
We're not using EJBs and we don't need ThreadLocal variables. OFbiz has the Delegator, and as my code fragment shows, if the framework needs to determine what transaction is currently running, it fetches the transaction from the Delegator. Adrian Crum Sandglass Software www.sandglass-software

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Jacopo Cappellato
This is an interesting conversation and I don't have yet a strong opinion, and I am happy to discuss new ideas, but I would be careful in changing the behavior of this core part of the system because the way transactions are managed in OFBiz has proven to be good and well suited for most of the

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Adrian Crum
That doesn't solve the problem with using ThreadLocal variables. Adrian Crum Sandglass Software www.sandglass-software.com On 8/26/2014 11:06 PM, Scott Gray wrote: I don't think that's an issue with our transaction handling, it's simply a problem that the cache isn't transaction aware. If the

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Scott Gray
I don't think that's an issue with our transaction handling, it's simply a problem that the cache isn't transaction aware. If the cache were to be transaction aware it would need to implement the XAResource interface or perhaps even the simpler Synchronization interface and push cache entries t

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Adrian Crum
The concepts of "suspend" and "resume" are implemented by a ThreadLocal stack. A "suspend" pushes the current transaction on the stack, and a "resume" pops a transaction off the stack. If you read the Jira issue, I point out another problem with this clunky implementation - calling "commit" do

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Scott Gray
Okay so I guess I don't really understand what you're suggesting, or how it really differs much from what we have now. It's also not clear what your suggested API changes have to do with the ThreadLocal usages? Thanks Scott On 26/08/2014, at 3:22 pm, Adrian Crum wrote: > Just use the Delega

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Adrian Crum
Just use the Delegator factory. Adrian Crum Sandglass Software www.sandglass-software.com On 8/26/2014 2:43 PM, Scott Gray wrote: Hi Adrian, I'll probably have plenty of questions, but the first that comes to mind is: how would you use a delegator outside of a transaction with this approach?

Re: Discussion: Entity Engine Redesign

2014-08-26 Thread Scott Gray
Hi Adrian, I'll probably have plenty of questions, but the first that comes to mind is: how would you use a delegator outside of a transaction with this approach? Thanks Scott On 25/08/2014, at 10:51 am, Adrian Crum wrote: > One persistent problem with the current Entity Engine implementatio

Discussion: Entity Engine Redesign

2014-08-25 Thread Adrian Crum
One persistent problem with the current Entity Engine implementation is the use of ThreadLocal variables in the Delegator and Transactions. Their use makes it difficult (and sometimes impossible) to fix Entity Engine bugs. They also make it impossible to multi-thread a Delegator instance. Her