Bugs item #590816, was opened at 2002-08-04 17:37 Message generated for change (Comment added) made by ejort You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=376685&aid=590816&group_id=22866
Category: JBossCX Group: CVS HEAD >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Larry Sanderson (lsanders) >Assigned to: Adrian Brock (ejort) Summary: XAProto Errors on closed XA Resources Initial Comment: Symptoms: An XA Resource (any - JDBC, JMS, etc..) is associated with a global transaction (tx1). That resource is closed, and put back in the free pool, but tx1 remains open. Another client, within another transaction (tx2) obtains the resource from the free pool. While the resource is associated with tx2, tx1 attempts to commit or roll-back. Result: XAException (errorCode=XAER_PROTO). Since most transactios are relatively short lived, and the pool is usually sufficiently large, it is rare for a connection to be used by a second client before the first transaction rolls-back or commits. Therefore this bug does not appear except under relatively heavy load. I have tracked the bug down to a change made 6 weeks ago on org.jboss.resource.connectionmanager.XATxConnection Manager. When an XAResource is is closed, it is first delisted with tha XA flag XAResource.TMSUSPEND. Because it is suspended subsequent commits and rollbacks are forced to first resume that transaction, then end it successfully (ie XAResource.start (TMRESUME); XAResource.end(TMSUCCESS)). (actually the JTA and XA/Open spec differ here: JTA requires both calls, but XA/Open only requires the second). But, if that resource has already been enlisted into another transaction then it is a protocol error (XAER_PROTO) to attempt to resume the old transaction. Prior to 6 weeks ago, XAConnections were delisted with the XA flag XAResource.TMSUCCESS. This allowed the resources to be committed and rolled back without first re-enlisting that resource, and avoiding these protocol errors. According to the cvs comments on XATxConnectionManager.java, this flag was switched to make SwiftMQ work correctly. I believe that this flag should be switched back to its original state - TMSUCCESS is the proper flag to use when a user requests to close a transaction. If this is not done, then at the least these connections should not be allowed back into the free pool until their corresponding transactions have been closed. -Larry ---------------------------------------------------------------------- >Comment By: Adrian Brock (ejort) Date: 2003-09-15 10:38 Message: Logged In: YES user_id=9459 It is the managed connection that is pooled. If the managed connection cannot handle mutiple uncommitted XAResources (not running concurrently) you should use <track-connection-by-tx> Regards, Adrian ---------------------------------------------------------------------- Comment By: Jason Dillon (user57) Date: 2002-08-07 21:41 Message: Logged In: YES user_id=15045 I am guessing that you have the information that you need to look into this further? Let me know otherwise. --jason ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-07 21:06 Message: Logged In: YES user_id=379453 Agreed - I will not make any changes until we get Jason's input. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-08-07 20:14 Message: Logged In: NO I don't agree. You should at least talk with Jason why he changed the code. Probably he is on vacation or whatever, however, he is very active here and will be back. I've forwarded this issue to him and I'm sure you'll hear from him. If that change is an urgent problem for you, just make your private code change for the meantime and use it until it is clearified. I'm watching this bug as well so I can help if required. -- Andreas ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-07 15:41 Message: Logged In: YES user_id=379453 I have not been able to contact Jason. I will take a stab at reproducing his original problem, but if I can't, and I don't hear any arguments about why this change was made in the first place, then I will change TMSUSPEND back to TMSUCCESS. ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-06 15:22 Message: Logged In: YES user_id=379453 Thanks Andreas - you have helped a lot. Next stop: Jason. Mr. Dillon, you out there? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-08-06 15:10 Message: Logged In: NO The whole thing refers to this thread on [EMAIL PROTECTED]: http://www.jguru.com/forums/view.jsp?EID=914840 See the *last* message. You may ask Jason Dillon about the problems he had. Let me know whether there is anything I can help. -- Andreas ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-06 14:58 Message: Logged In: YES user_id=379453 OK - I misinterpreted your previous comment: "You must suspend xid1 before working with xid2". The code you gave explains it perfectly - thanks. So, back to the original point of this bug report: On the close of a resource why is it necessary to delist with TMSUSPEND instead of TMSUCCESS? The code Andreas just supplied indicates that SwiftMQ would have no problems with TMSUCCESS, but the comments in cvs indicate that it was SwiftMQ that instigated the change. Any thoughts? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-08-06 14:33 Message: Logged In: NO Well, the code snipped from the JTA spec p. 19 isn't complete; the flag is missed in XAResource.end(). I have tested it just here with SwiftMQ and just this flag makes the difference. Here's the test that works: <code> public void testNP() { try { Xid xid1 = new XidImpl(); xares.start(xid1, XAResource.TMNOFLAGS); TextMessage msg = qs.createTextMessage(); for (int i = 0; i < 2; i++) { msg.setText("Msg1: " + i); sender.send(msg, DeliveryMode.NON_PERSISTENT, Message.DEFAULT_PRIORITY, Message.DEFAULT_TIME_TO_LIVE); } xares.end(xid1, XAResource.TMSUCCESS); Xid xid2 = new XidImpl(); xares.start(xid2, XAResource.TMNOFLAGS); for (int i = 0; i < 3; i++) { msg.setText("Msg2: " + i); sender.send(msg, DeliveryMode.NON_PERSISTENT, Message.DEFAULT_PRIORITY, Message.DEFAULT_TIME_TO_LIVE); } xares.prepare(xid1); xares.commit(xid1, false); xares.end(xid2, XAResource.TMSUCCESS); xares.prepare(xid2); xares.commit(xid2, false); } catch (Exception e) { e.printStackTrace(); fail("test failed: " + e); } } </code> However, if you end xid1 with TMSUSPEND like <code> xares.end(xid1, XAResource.TMSUSPEND); </code> SwiftMQ will throw an XAException which is compliant with the JTA spec as it states: "If TMSUSPEND is specified in flags, the transaction branch is temporarily suspended in incomplete state. The transaction context is in suspened state and must be resumed via start with TMRESUME specified." (P. 41) So IMO SwiftMQ acts compliant and it depends very much on the flag you use at XAResource.end(). -- Andreas ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-06 13:24 Message: Logged In: YES user_id=379453 Sorry, I should have been more descriptive. This is from the JTA spec (p. 19): <code> // Suppose we have some transactional connection-based // resource r1 that is connected to an enterprise information // service system. // XAResource xares = r1.getXAResource(); xares.start(xid1); // associate xid1 to the connection .. xares.end(xid1); // dissociate xid1 to the connection .. xares.start(xid2); // associate xid2 to the connection .. // While the connection is associated with xid2, // the TM starts the commit process for xid1 status = xares.prepare(xid1); .. xares.commit(xid1, false); </code> If SwiftMQ does not support this, then it is not JTA compliant. I'm not sure if I'm making myself clear - please let me know if I can provide any more info. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-08-06 12:48 Message: Logged In: NO Huh? Why is SwiftMQ not JTA compliant? Please explain. -- Andreas ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-06 12:44 Message: Logged In: YES user_id=379453 OK, so it appears that SwiftMQ is not JTA compliant. How best to procede? I suggest we change TMSUSPEND back to TMSUCCESS so that compliant resources work without this subtle bug, and come up with some other approach to tackle non-compliant resources. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-08-06 07:25 Message: Logged In: NO Yes. You must suspend xid1 before working with xid2. You cannot work in parallel with both xids without suspend/resume. Keep in mind that a JMS session (XA or not) has a single transaction context at a time, hence, you must load/unload (with suspend/resume) the tx into the session if you have multiple tx on our XAResource. -- Andreas ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-05 23:19 Message: Logged In: YES user_id=379453 OK - it sounds JTA compliant so far. The question that I have is: While the XASession is enlisted within transaction, tx1, (via XAResource.start(xid1 xx)), can a second transaction, tx2, be committed (via XAResource.commit(xid2)). Look at the sample code in the JCA spec, p. 19. A JTA compliant XA resource must be support this style of call for commit, rollback, prepare, and forget. If this is supported, then using XAResource.end(TMSUCCESS) should work perfectly when closing the sessions. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2002-08-05 22:16 Message: Logged In: NO In SwiftMQ an XASession has a single tx at a time. If you suspend a XA tx, the session switches automatically to a local tx (in fact, you can use the local tx then). You can have multiple XA tx in suspended state. To activate it, you have to resume it and then you can end it. That's the background on the XAER_PROTO (end is either called when another XA tx or a local tx is active). -- Andreas Mueller (SwiftMQ) ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-05 16:26 Message: Logged In: YES user_id=379453 Thanks for your response Ole, but the problem I am speaking of exists whether or not you re-enlist the resource. The conflict in the JTA spec is only on the call to start (TM_RESUME) - the state diagram says it is unnecessary, the text say that it is required. Both, however, require a call to end(TM_SUCCESS). (In fact the XA/Open spec says on p. 32: "[the committed transaction] must have been ended by using xa_end() with TMSUCCESS ...") This would still be a bug in the current XA Connection Pooling, since an attempted commit on tx1 would effectively end tx2's transaction. (This would lead to an XAER_NOTA on the next use of tx1). The root problem is that the closed connection is suspended instead of ended successfully. ---------------------------------------------------------------------- Comment By: Ole Husgaard (sparre) Date: 2002-08-05 15:48 Message: Logged In: YES user_id=175257 The problem here is that JTA 1.0.1 contradicts itself. So we have to chose if we should comply with the state tables at page 17-18, or if we should comply with the text at page 41. Originally, I decided to comply with the state tables, since that made most sense to me. After all, we may not always know if the resource needs to do more work on behalf of the tx when we delist, and a temporary re-enlist at commit time has the problems we observe here. But that broke the XA wrapper, so I changed the TM code to what it is now. I wouldn't mind changing back, so that resources are not temporarily re-enlisted during commit. However, that may break some XA resources. The XA wrapper is the only XAResource that I know has this problem, and since we have the source for our wrapper, we can fix that. But there may be other resources having the same problem as the wrapper (though unlikely)... Best Regards, Ole Husgaard. ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-04 21:15 Message: Logged In: YES user_id=379453 > Another possibility would be for the tx manager to > synchronize by xid attempts to do work on a XAResource, > so if it wanted to commit using an XAResource that was > currently enrolled in another tx, it would just wait until > it became available. This would require the tx manager to > track the state of all XAResources it knew anything about: > currently we don't have or need anything like this. The Informix XA drivers already do this, which introduces an inisdious little deadlock bug. Assume tx1 has done work and locked a row, then closed the connecton. tx2 uses that connection to also write to that row, but waits for the lock to be released. Meanwhile, tx1 is trying to commit, but it can't get at the connection. To work around this, we had to wrap the Informix XA drivers, and intercept all calls to commit, rollback, prepare, and forget. These methods now are invoked on a small internal pool of connections that are not part of the main JBoss pool. Needless to say, I do not recommend this solution. -Larry ---------------------------------------------------------------------- Comment By: Larry Sanderson (lsanders) Date: 2002-08-04 21:07 Message: Logged In: YES user_id=379453 I don't quite follow why SwiftMQ doesn't like TM_SUCCESS. If you need to enlist another SwiftMQ connection, then it should use a different BranchQualifier since SwiftMQ always returns false on isSameRM(). It seems to me like TM_SUCCESS is the correct thing to use here - I can't think of any example where a JTA compliant resource would have a problem with that. (Plus, now prior to each commit and rollback, the TM is forced to call an extra start and end. This can't be good for performance) Please, let me know if I am missing something? ---------------------------------------------------------------------- Comment By: David Jencks (d_jencks) Date: 2002-08-04 20:23 Message: Logged In: YES user_id=60525 I think this clearly shows that the xa protocol is pretty much ludicrous to try to use in a multithreaded pooled environment. I don't think we can go back to TM_SUCCESS, I think we then couldn't do more work in that branch on that RM. (I think this was the problem with SwifMQ) I also don't like firmly tieing the tx to the connection/XAResource: this eliminates the scalability advantages of xa over local tx. I think it would be a good idea however to loosely tie the tx and the connection: when a connection handle is closed, the ManagedConnection could be put in a pool of "partly in use" connections, and if the main pool ran out, one of these could be delisted from its current tx and used in the new request. Another possibility would be for the tx manager to synchronize by xid attempts to do work on a XAResource, so if it wanted to commit using an XAResource that was currently enrolled in another tx, it would just wait until it became available. This would require the tx manager to track the state of all XAResources it knew anything about: currently we don't have or need anything like this. Comments? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=376685&aid=590816&group_id=22866 ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
