Hi Paul Problem with that, it's all or nothing. We only need this for specific use cases. Having forUpdate in EntityQuery gives us that flexibility. It could be changed to use either method which depends on the scenario you use it for Gareth Carter Software Development Analyst Stannah Management Services Ltd IT Department Ext: 7036 DDI: 01264 364311
Please consider the environment before printing this email. -----Original Message----- From: Paul Foxworthy [mailto:p...@cohsoft.com.au] Sent: 27 October 2017 12:42 AM To: dev@ofbiz.apache.org Subject: Re: Updates to EntityQuery Hi all, As far as I know, FOR UPDATE would only break SQLite, and I can live with that. Gareth, would optimistic instead of pessimistic work for your situation? The database does less of the work and it's more scalable. I haven't tested it out, but OFBiz provides optimistic locking if you turn on enable-lock. Cheers Paul Foxworthy On 27 October 2017 at 03:49, Jacques Le Roux <jacques.le.r...@les7arts.com> wrote: > I agree with Scott and Taher, > > About your question Gareth, we once had a hard coded "for update" > somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ). > I put it in. Then Jacopo crossed an issue in a cluster and rightly > changed it to something better See > https://issues.apache.org/jira/browse/OFBIZ-2353 for more where Adrian > gave some directions also. > > Since I did not read all in the thread and there is no patch yet I > don't know if something alike applies to EntityQuery API. Putting a > for update is certainly much simpler, so just saying for now. > > Jacques > > > > Le 26/10/2017 à 10:46, Gareth Carter a écrit : > >> HI all >> >> Thank for the response >> >> It could be considered a hint, if the underlying db does not support >> it then don't use it. Adding an attribute on the datasource could be >> the way to go but I also believe we need methods to tell the system >> when to use it, you don't need it all the time only in selective cases. >> >> Until support is added I see ofbiz as purely RDMS. I don't believe >> ofbiz should restrict itself on the assumption it might support >> different databases such as nosql. >> >> There could be an alternative to forUpdate, anyone considered >> implementing a distributed locking mechanism? The whole point for >> using forUpdate is to prevent one or more code blocks changing the >> same database records concurrently, db row locks help but are not >> supported by all databases but an external locking mechanism may >> provide a generic way of dealing with this. OFBiz already provides a >> semaphore on services but I believe you need something at entity >> level Gareth Carter Software Development Analyst Stannah Management >> Services Ltd IT Department >> Ext: >> 7036 >> DDI: >> 01264 364311 >> >> >> Please consider the environment before printing this email. >> >> -----Original Message----- >> From: Taher Alkhateeb [mailto:slidingfilame...@gmail.com] >> Sent: 26 October 2017 6:13 AM >> To: OFBIZ Development Mailing List <dev@ofbiz.apache.org> >> Subject: Re: Updates to EntityQuery >> >> Hmmm usually my first gut reaction would be "keep things consistent", >> however Scott made a good argument for supporting non-common DB features. >> So my suggestion would be to use something like the widget's >> <platform-specific ...> tag (or equivalent in code) for any features >> not supported in all DBMSs. By labeling it as such the user is aware >> that the DB selection is restricted. >> >> Now with regards to your comment about NoSQL, such databases are >> fundamentally different, and act more like a key-value based entities >> without a schema or the resulting ACID properties. It is possible to >> make it work with the entity engine, but then you'd lose the >> flexibility it gives you because you want to restrict yourself to the >> relational model that we apply in OFBiz. The idea of NoSQL is to >> think at a different conceptual level of your domain model. >> >> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray >> <scott.g...@hotwaxsystems.com> >> wrote: >> >>> I think it's better to have for-update available for use for the >>> majority of use-cases than it is to exclude it altogether. I'd >>> rather lose the databases that don't support it than constrain the >>> ones that do and are used most by pretty much everyone. It would be >>> nice to hear from anyone using a database that doesn't support it >>> since I'm just guessing most of us use postgres or mysql. >>> >>> Has anyone actually looked at how NoSQL databases might work with >>> the OFBiz entity engine or are the concepts too far removed from SQL >>> for it to even be workable? >>> >>> If we did want to support for-update globally, we could always have >>> the delegator simulate it by issuing an UPDATE to lock the rows >>> prior to performing the SELECT. Although it might not work >>> perfectly in all cases, it could be a good compromise. e.g. >>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- >>> lock the rows SELECT * FROM table WHERE ...conditions -- select the >>> rows >>> >>> Regards >>> Scott >>> >>> >>> >>> On 26 October 2017 at 13:45, Paul Foxworthy <p...@cohsoft.com.au> wrote: >>> >>> Hi Gareth, >>>> >>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't >>>> support >>>> it: >>>> >>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >>>> https://sqlite.org/lang_select.html >>>> >>>> FOR UPDATE only makes sense for relational databases. Adding >>>> assumptions it's available would make it harder to use a NoSQL >>>> database in future. >>>> >>>> In the case of SQLite, in effect it's got serialized isolation >>>> anyway, so the FOR UPDATE is not strictly necessary. >>>> >>>> I suggest OFBiz shouldn't have a method that generates >>>> syntactically incorrect SQL for any database we want to support, >>>> and we claim to support several. Where there are variations between >>>> SQL dialects, we have attributes in the data source, such as "join-style". >>>> >>>> So think our choices are: >>>> >>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by >>>> a data source. OFBiz would only add the "FOR UPDATE" if it is >>>> supported. In the case of SQLite, I suspect we'd get away without >>>> it, but there may be other databases where the absence of "FOR UPDATE" >>>> might bring on unexpected behaviour. >>>> >>>> 2. Don't provide a method to expose FOR UPDATE. >>>> >>>> Could you achieve the results you want with a higher transaction >>>> isolation level like repeatable read, or with optimistic locking >>>> (enable-lock)? >>>> >>>> What do you think? >>>> >>>> Cheers >>>> >>>> Paul Foxworthy >>>> >>>> >>>> On 26 October 2017 at 07:50, Gareth Carter >>>> <gareth.car...@stannah.co.uk> >>>> wrote: >>>> >>>> Ok I will create a Jira with patch for review. >>>>> >>>>> >>>>> >>>>> Sent from my Samsung Galaxy smartphone. >>>>> >>>>> >>>>> >>>>> Gareth Carter >>>>> >>>>> >>>>> Software Development Analyst >>>>> >>>>> >>>>> Stannah Management Services Ltd >>>>> >>>>> >>>>> IT Department >>>>> >>>>> >>>>> Ext: >>>>> >>>>> >>>>> 7036 >>>>> >>>>> >>>>> DDI: >>>>> >>>>> >>>>> 01264 364311 >>>>> >>>>> >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>> environment before printing this email. >>>>> >>>>> >>>>> -------- Original message -------- >>>>> From: deepak nigam <deepak.nigam1...@gmail.com> >>>>> Date: 25/10/2017 14:06 (GMT+00:00) >>>>> To: dev@ofbiz.apache.org >>>>> Subject: Re: Updates to EntityQuery >>>>> >>>>> Looking good to me also. Let me know if I can be of any help. >>>>> >>>>> >>>>> Thanks & Regards >>>>> -- >>>>> Deepak Nigam >>>>> >>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >>>>> yash.sha...@hotwaxsystems.com> >>>>> wrote: >>>>> >>>>> I am all up for it, please let me know if I could help. >>>>>> >>>>>> Thanks & Regards, >>>>>> -- >>>>>> *Pradhan Yash Sharma* >>>>>> *HotWax Systems* | >>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>> >>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >>>>>> yash.sha...@hotwaxsystems.com> >>>>>> wrote: >>>>>> >>>>>> Yes, Usage of the stream will surely enhance performance to a >>>>>>> certain extent and removes ceremony from the code base. I think >>>>>>> Parallel >>>>>>> >>>>>> streams >>>>> >>>>>> will add enhancements in many folds as we are using a multicore >>>>>>> >>>>>> processor >>>>> >>>>>> (I have not tested yet), but the Functional approach seems promising. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> -- >>>>>>> *Pradhan Yash Sharma* >>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http:// >>>>>>> >>>>>> www.hotwaxsystems.com> >>>> >>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >>>>>>> gareth.car...@stannah.co.uk> wrote: >>>>>>> >>>>>>> forUpdate >>>>>>>> >>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with >>>>>>>> >>>>>>> shorthand >>>>> >>>>>> methods in EntityQuery to enable. We then made a change to >>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" >>>>>>>> on the >>>>>>>> >>>>>>> end >>>>> >>>>>> of the SQL select statement - this allows for DB row locks (we >>>>>>>> use >>>>>>>> >>>>>>> postgres >>>>>> >>>>>>> and works but have not tested other databases). I believe >>>>>>>> there may >>>>>>>> >>>>>>> have >>>>> >>>>>> been a discussion about this before >>>>>>>> >>>>>>>> >>>>>>>> forEach on EntityQuery >>>>>>>> >>>>>>>> Use Consumer in java and groovy to iterate over a query. This >>>>>>>> can >>>>>>>> >>>>>>> reduce >>>>> >>>>>> memory consumption (replacement for queryList()) and boiler >>>>>>>> plate >>>>>>>> >>>>>>> code >>>> >>>>> (eg >>>>>> >>>>>>> queryIterator(), while loop and close) >>>>>>>> >>>>>>>> Example: >>>>>>>> >>>>>>>> EntityQuery.use(delegator).from("Foobar") >>>>>>>> .forEach(item -> >>>>>>>> Debug.logInfo(item.toString(), module); ); >>>>>>>> >>>>>>>> A further update could be to provide stream capabilities >>>>>>>> >>>>>>>> Hope this helps, I can provide a patch aswell >>>>>>>> >>>>>>>> Gareth Carter >>>>>>>> Software Development Analyst >>>>>>>> Stannah Management Services Ltd IT Department >>>>>>>> Ext: >>>>>>>> 7036 >>>>>>>> DDI: >>>>>>>> 01264 364311 >>>>>>>> >>>>>>>> >>>>>>>> Please consider the environment before printing this email. >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Arun Patidar [mailto:arun.pati...@hotwaxsystems.com] >>>>>>>> Sent: 25 October 2017 5:49 AM >>>>>>>> To: dev@ofbiz.apache.org >>>>>>>> Cc: arunpati...@apache.org >>>>>>>> Subject: Re: Updates to EntityQuery >>>>>>>> >>>>>>>> Hello Gareth, >>>>>>>> >>>>>>>> Please provide some more details or patch to understand - >>>>>>>> forUpdate >>>>>>>> >>>>>>> and >>>>> >>>>>> forEach method utility. >>>>>>>> >>>>>>>> getFieldMap method looks good to me. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks & Regards >>>>>>>> --- >>>>>>>> Arun Patidar >>>>>>>> Manager, Enterprise Software Development >>>>>>>> >>>>>>>> HotWax Systems Pvt Ltd. >>>>>>>> >>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >>>>>>>> gareth.car...@stannah.co.uk> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi all >>>>>>>>> >>>>>>>>> We have internally patched EntityQuery with some additional >>>>>>>>> functionality and before I create a Jira was going to see what >>>>>>>>> the >>>>>>>>> >>>>>>>> community thinks. >>>>>>>> >>>>>>>>> New functionality: >>>>>>>>> >>>>>>>>> - New method getFieldMap which returns a Map object of >>>>>>>>> >>>>>>>> selected >>>>> >>>>>> fields from GenericValue objects, useful for creating cache map >>>>>>>>> objects for lookup >>>>>>>>> >>>>>>>>> - Support forUpdate >>>>>>>>> >>>>>>>>> - forEach method to accept Consumer >>>>>>>>> >>>>>>>>> We have found these useful and believe the project can >>>>>>>>> benefit, >>>>>>>>> >>>>>>>> let >>>> >>>>> me >>>>> >>>>>> know what you think >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Gareth Carter >>>>>>>>> >>>>>>>>> >>>>>>>>> Software Development Analyst >>>>>>>>> >>>>>>>>> >>>>>>>>> Stannah Management Services Ltd >>>>>>>>> >>>>>>>>> >>>>>>>>> IT Department >>>>>>>>> >>>>>>>>> >>>>>>>>> Ext: >>>>>>>>> >>>>>>>>> >>>>>>>>> 7036 >>>>>>>>> >>>>>>>>> >>>>>>>>> DDI: >>>>>>>>> >>>>>>>>> >>>>>>>>> 01264 364311 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>>>>>> >>>>>>>> environment >>>>>> >>>>>>> before printing this email. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> This email is intended only for the above addressee. It may >>>>>>>>> >>>>>>>> contain >>>> >>>>> privileged information. If you are not the addressee you must not >>>>>>>>> copy, distribute, disclose or use any of the information in >>>>>>>>> it. If >>>>>>>>> >>>>>>>> you >>>>> >>>>>> have received it in error, please delete it and notify the sender. >>>>>>>>> >>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah >>>>>>>>> >>>>>>>> Management >>>> >>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd >>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No. >>>>>>>>> >>>>>>>> 964804, >>>>> >>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd >>>>>>>>> registered >>>>>>>>> >>>>>>>> No. >>>>> >>>>>> 02452728. >>>>>>>> >>>>>>>>> All registered offices at Watt Close, East Portway, Andover, >>>>>>>>> Hampshire, >>>>>>>>> SP10 3SD, England. >>>>>>>>> >>>>>>>>> All Registered in England and Wales. >>>>>>>>> >>>>>>>>> This message has been scanned for malware by Websense. >>>>>>>>> www.websense.com<http://www.websense.com> >>>>>>>>> >>>>>>>>> >>>>>>>> To report this email as spam, please send the original message, >>>>>>>> >>>>>>> complete >>>>> >>>>>> with headers to s...@websense.com >>>>>>>> >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 >>>> Australia >>>> >>>> Phone: +61 3 9585 6788 >>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxN >>>> CsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A >>>> -OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSA >>>> Ux33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z >>>> Email: i...@coherentsoftware.com.au >>>> >>>> > -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicY2BnsD7JwLD1IANDUU6lkXGiXnFRmV5uYmZOcn5eSVF-jl5yfi5DmWWgX3pwiLmBgZmhhQmDe2JRakmGnnNiUUlqkUNxSWJeXmIGUKFeaTZDRklJgZW-fnl5OVAgI7UoNa-kOD-tpByoBWSUXmKpPgMDw5sdDAwAvaYpRA&Z Email: i...@coherentsoftware.com.au
![http://logos.stannah.co.uk/enviro.jpg]Please](http://logos.stannah.co.uk/enviro.jpg]Please){kind=link}