[JBoss-dev] [JBossCache] - Re: Transactions/Locking Across VMs Issues
#1 exists() doesn't acquire a lock, use get(). Here's the javadoc: /** * Checks whether a given node exists in the tree. Does not acquire any locks in doing so (result may be dirty read) * @param fqn The fully qualified name of the node * @return boolean Whether or not the node exists * @jmx.managed-operation */ #2: you try to update the same data on 3 nodes concurrently. Because you use REPL_SYNC, TX commit runs a two phase commit protocol, which needs to lock all data on all 3 nodes. However, because other TXs on those nodes already hold on to the locks, in most cases you will time out attempting to acquire the lock, and then rolling back your TX. This is a typical distributed deadlock. If one TX manages to acquire all locks on all 3 servers before timing out, b/c the other TXs rolled back, then that one TX will succeed. So the chance to succeed is that small timeframe where no other TX holds a lock. Of course, the more servers access the same data, the smaller the timeframe. Bela View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3854977#3854977 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3854977 --- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] [JBossCache] - Re: Transactions/Locking Across VMs Issues
"bela" wrote : #1 exists() doesn't acquire a lock, use get(). We tried the top-level if with both and exist and get to see if it made any difference before I posted, because we thought this might be the case. Thanks for (correctly) pointing back to the Javadoc. "bela" wrote : #2: you try to update the same data on 3 nodes concurrently. Because you use REPL_SYNC...This is a typical distributed deadlock. Thanks for the distributed deadlock explanation. I knew that there was something I was missing. For us, the locking is not so important (so we will probably not use transactions). What are the solutions for this type of problem? Random sleep times before retrying if a timeout occurs? What about a master node that hands out locks/blocks when called? It seems like there would be a pretty standard set of possibilities when this type of situation is encountered. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3855017#3855017 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3855017 --- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] [JBossCache] - Re: Transactions/Locking Across VMs Issues
By refactoring the above code to retry a configurable number of times and by
randomizing the time to wait between retries, I was able to get the behavior I
expected. Essentially, this code takes a concurrent event, and makes it not
concurrent (avoiding the distributed deadlock situation altogether).
The refactored code:
| public void test()
| throws Exception
| {
| _cache.startService();
|
| try
| {
|
| // Try to insert into the cache a maximum of 2 times.
| int retryCount = 2;
|
| for (int i = 0; i < retryCount; i++)
| {
| try
| {
| putInCache("key", _value);
| }
| catch (AlreadyBound ab)
| {
| throw ab;
| }
| catch (Exception e)
| {
| System.out.println();
| System.out.println("= Try #" + i + "
=");
| e.printStackTrace(System.out);
|
System.out.println("==");
| System.out.println();
|
| // Wait at least 100 milliseconds.
| Thread.sleep(_random.nextInt(4900) + 100);
|
| continue;
| }
|
| break;
| }
| }
| catch (Throwable t)
| {
| t.printStackTrace(System.out);
| System.out.println();
| }
| finally
| {
| System.out.println();
| System.out.println("Done.");
| System.out.println();
| }
| }
|
| private void putInCache(String key, String value)
| throws Exception
| {
| UserTransaction tx =
| new DummyUserTransaction(DummyTransactionManager.getInstance());
|
| tx.begin();
|
| Object previous = _cache.put("aom", "key", _value);
|
| if (previous != null)
| {
| tx.rollback();
|
| throw new AlreadyBound();
| }
| else
| {
| tx.commit();
| }
| }
|
Anyone else have other ideas on how to handle distributed deadlock situations?
View the original post :
http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3855027#3855027
Reply to the post :
http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3855027
---
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
___
JBoss-Development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
[JBoss-dev] [JBossCache] - Re: Transactions/Locking Across VMs Issues
JBoss itself (the appserver) uses random sleeps and retries (up to 5 times). Unless you have distributed deadlock detection, this is the only way to recover from deadlocks. How to avoid this ? Work on separate data sets in different nodes, e.g. node1 works on data 1-10, node2 on 11-20, node 3 on 21-30 etc. Sometimes this is easy to implement, e.g. in HTTP session replication you can enable session stickiness. Bela View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3855155#3855155 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3855155 --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
