Re: does commitChanges lock all other queries?

Tue, 12 Dec 2006 06:32:41 -0800 

yeah, I am all for using multiversion cc as much as possible.
Do we need to lock the DataRowStore for the duration of the commit? Would it be possible to only update the cache after the db-commit are finished and let the DataRowStore return old data while committing?
Good question. Maybe I am still thinking in terms of Cayenne 1.1 where the lock was absolutely needed.
On commit lock spans the following chain of operations: update query preparation -> DB commit -> cache postprocessing. Maybe we can reduce the scope to just the last step, resulting in some fuzziness in the baseline snapshot versions.
Are there any brave souls with high volume update-heavy sites to try this modification in production to see how much damage it causes? Tore, do you want to try it in your app? I am asking cause it may take me a while to do thorough stress testing, although just moving the lock is trivial.
What are the status of "I think we will be changing cache format from DataRows (that cache DB data) to some kind of object property- based "rows""? http://www.objectstyle.org/cayenne/lists/cayenne-devel/ 2006/04/0248.html
haven't started to look at it yet, but this is something worth investigating on the long run. 

Andrus



On Dec 12, 2006, at 3:55 PM, Tore Halset wrote:


Hello.

I have moved the thread to cayenne-dev.
This is interesting. It reminds me of the different ways to concurrency control in a database :) PostgreSQL MVCC being one of the best. 
http://www.postgresql.org/docs/current/static/mvcc-intro.html
Do we need to lock the DataRowStore for the duration of the commit? Would it be possible to only update the cache after the db-commit are finished and let the DataRowStore return old data while committing?
What are the status of "I think we will be changing cache format from DataRows (that cache DB data) to some kind of object property- based "rows""? http://www.objectstyle.org/cayenne/lists/cayenne-devel/ 2006/04/0248.html 

 - Tore.

On Dec 12, 2006, at 13:21 , Andrus Adamchik wrote:
* On commit, DRS is locked for the duration of the query execution (DataDomainFlushAction.flush()) * On select, DRS is locked AFTER the query for the duration of the row-to-object conversion 

So yes, slow commits will block object queries.
Hmm... I wonder if we can do cache update on select in a thread separate from the calling thread (via EventManager?), and avoid locking during object creation..? This should generally improve throughput. Something to think about.
Alternatively for slow commits you can try to use a special DataContext that uses its own local instance of DataRowStore (passed in ObjectStore constructor), thus avoiding a shared lock. 

Andrus


On Dec 12, 2006, at 1:48 PM, Tore Halset wrote:

Hello.
One of our applications blocked for some time today. It worked out fine after a couple of minutes. It is running cayenne-1.2. We will upgrade to 2.0.1 in the start of january. The attached stacktraces are extracted using JBoss jmx-console.
Thread http-0.0.0.0-80-6 are performing a slow DataContext.commitChanges. It locks our MS SQL Server for some minutes. I guess we will be able to optimize this by deleting some old records or creating a index.
Some other threads are performing simple queries, but are blocked by a lock in DataRowStore. Are all (other) queries blocked while doing a commitChanges? I have seen some discussion around locks in DataRowStore lately, but I do not know if this is related. 

 - Tore.
Thread: http-0.0.0.0-80-3 : priority:5, demon:true, threadId:38, threadState:BLOCKED, threadLockName:[EMAIL PROTECTED] org.objectstyle.cayenne.access.DataRowStore.getCachedSnapshot (DataRowStore.java:387) org.objectstyle.cayenne.access.DataDomainQueryAction.interceptOIDQue ry(DataDomainQueryAction.java:164) org.objectstyle.cayenne.access.DataDomainQueryAction.execute (DataDomainQueryAction.java:142) org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java: 766) org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery (ObjectContextQueryAction.java:253) org.objectstyle.cayenne.access.DataContextQueryAction.execute (DataContextQueryAction.java:90) org.objectstyle.cayenne.access.DataContext.onQuery (DataContext.java:1422) org.objectstyle.cayenne.access.DataContext.performQuery (DataContext.java:1411) org.objectstyle.cayenne.DataObjectUtils.objectForQuery (DataObjectUtils.java:290) org.objectstyle.cayenne.DataObjectUtils.objectForPK (DataObjectUtils.java:276) org.objectstyle.cayenne.DataObjectUtils.objectForPK (DataObjectUtils.java:185)
Thread: http-0.0.0.0-443-2 : priority:5, demon:true, threadId:47, threadState:BLOCKED, threadLockName:[EMAIL PROTECTED] org.objectstyle.cayenne.access.ObjectResolver.synchronizedObjectsFro mDataRows(ObjectResolver.java:133) org.objectstyle.cayenne.access.DataDomainQueryAction.interceptObject Conversion(DataDomainQueryAction.java:373) org.objectstyle.cayenne.access.DataDomainQueryAction.execute (DataDomainQueryAction.java:151) org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java: 766) org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery (ObjectContextQueryAction.java:253) org.objectstyle.cayenne.access.DataContextQueryAction.execute (DataContextQueryAction.java:90) org.objectstyle.cayenne.access.DataContext.onQuery (DataContext.java:1422) org.objectstyle.cayenne.access.DataContext.performQuery (DataContext.java:1411) org.objectstyle.cayenne.access.ToManyList.resolvedObjectList (ToManyList.java:343) 
org.objectstyle.cayenne.access.ToManyList.size(ToManyList.java:296)
Thread: EventDispatchThread-0 : priority:5, demon:true, threadId: 54, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: EventDispatchThread-1 : priority:5, demon:true, threadId: 55, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: EventDispatchThread-2 : priority:5, demon:true, threadId: 56, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: EventDispatchThread-3 : priority:5, demon:true, threadId: 57, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: EventDispatchThread-4 : priority:5, demon:true, threadId: 58, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: EventDispatchThread-0 : priority:5, demon:true, threadId: 59, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: EventDispatchThread-1 : priority:5, demon:true, threadId: 60, threadState:WAITING, threadLockName:java.util.Collections [EMAIL PROTECTED] 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.objectstyle.cayenne.event.EventManager$DispatchThread.run (EventManager.java:486)
Thread: http-0.0.0.0-443-15 : priority:5, demon:true, threadId: 104, threadState:BLOCKED, threadLockName:[EMAIL PROTECTED] org.objectstyle.cayenne.access.ToManyList.resolvedObjectList (ToManyList.java:332) 
org.objectstyle.cayenne.access.ToManyList.size(ToManyList.java:296)
Thread: http-0.0.0.0-80-6 : priority:5, demon:true, threadId:113, threadState:RUNNABLE, threadLockName:null 
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:129)
java.io.DataInputStream.readFully(DataInputStream.java:176)
java.io.DataInputStream.readFully(DataInputStream.java:152)
net.sourceforge.jtds.jdbc.SharedSocket.readPacket (SharedSocket.java:814) net.sourceforge.jtds.jdbc.SharedSocket.getNetPacket (SharedSocket.java:695) net.sourceforge.jtds.jdbc.ResponseStream.getPacket (ResponseStream.java:443) net.sourceforge.jtds.jdbc.ResponseStream.read(ResponseStream.java: 102) net.sourceforge.jtds.jdbc.ResponseStream.peek(ResponseStream.java: 87) 
net.sourceforge.jtds.jdbc.TdsCore.wait(TdsCore.java:3772)
net.sourceforge.jtds.jdbc.TdsCore.executeSQL(TdsCore.java:1031)
net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL (JtdsStatement.java:436) net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate (JtdsPreparedStatement.java:402) org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeUpda te(WrappedPreparedStatement.java:227) org.objectstyle.cayenne.access.jdbc.BatchAction.runAsIndividualQueri es(BatchAction.java:224) org.objectstyle.cayenne.access.jdbc.BatchAction.performAction (BatchAction.java:117) org.objectstyle.cayenne.dba.sqlserver.SQLServerBatchAction.performAc tion(SQLServerBatchAction.java:95) org.objectstyle.cayenne.access.DataNodeQueryAction.runQuery (DataNodeQueryAction.java:95) org.objectstyle.cayenne.access.DataNode.performQueries (DataNode.java:309) org.objectstyle.cayenne.access.DataDomainFlushAction.runQueries (DataDomainFlushAction.java:255) org.objectstyle.cayenne.access.DataDomainFlushAction.flush (DataDomainFlushAction.java:177) org.objectstyle.cayenne.access.DataDomain.onSyncFlush (DataDomain.java:830) org.objectstyle.cayenne.access.DataDomain$2.transform (DataDomain.java:801) org.objectstyle.cayenne.access.DataDomain.runInTransaction (DataDomain.java:856) org.objectstyle.cayenne.access.DataDomain.onSync(DataDomain.java: 798) org.objectstyle.cayenne.access.DataContext.flushToParent (DataContext.java:1261) org.objectstyle.cayenne.access.DataContext.commitChanges (DataContext.java:1165)
Thread: http-0.0.0.0-443-18 : priority:5, demon:true, threadId: 187, threadState:BLOCKED, threadLockName:[EMAIL PROTECTED] org.objectstyle.cayenne.access.ObjectResolver.synchronizedObjectsFro mDataRows(ObjectResolver.java:133) org.objectstyle.cayenne.access.DataDomainQueryAction.interceptObject Conversion(DataDomainQueryAction.java:373) org.objectstyle.cayenne.access.DataDomainQueryAction.execute (DataDomainQueryAction.java:151) org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java: 766) org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery (ObjectContextQueryAction.java:253) org.objectstyle.cayenne.access.DataContextQueryAction.execute (DataContextQueryAction.java:90) org.objectstyle.cayenne.access.DataContext.onQuery (DataContext.java:1422) org.objectstyle.cayenne.access.DataContext.performQuery (DataContext.java:1411)
Thread: http-0.0.0.0-443-19 : priority:5, demon:true, threadId: 188, threadState:BLOCKED, threadLockName:[EMAIL PROTECTED] org.objectstyle.cayenne.access.ObjectResolver.synchronizedObjectsFro mDataRows(ObjectResolver.java:133) org.objectstyle.cayenne.access.DataDomainQueryAction.interceptObject Conversion(DataDomainQueryAction.java:373) org.objectstyle.cayenne.access.DataDomainQueryAction.execute (DataDomainQueryAction.java:151) org.objectstyle.cayenne.access.DataDomain.onQuery(DataDomain.java: 766) org.objectstyle.cayenne.util.ObjectContextQueryAction.runQuery (ObjectContextQueryAction.java:253) org.objectstyle.cayenne.access.DataContextQueryAction.execute (DataContextQueryAction.java:90) org.objectstyle.cayenne.access.DataContext.onQuery (DataContext.java:1422) org.objectstyle.cayenne.access.DataContext.performQuery (DataContext.java:1411)

Reply via email to