Re: [Zope] session variables in the presence of conflicts
Dennis Allison wrote at 2005-11-18 13:15 -0800: >We are using MySQL but are fully transactional using innodb. >2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager >0x450431cc> at 1190763820> Thus, you should find out why "tpc_abort()" fails on the "ZMySQLDA.db.DB" instance... > ... >2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: >0x450431cc> at 1190763820> same here -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] session variables in the presence of conflicts
We are using MySQL but are fully transactional using innodb. The sort of problems we are seeing are (cruft removed) are things like: 2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager at 1190763820> 2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager at 1190774252> 2005-11-18T12:50:16 INFO ZODB conflict error at /courses/topics/navigation_box (55 conflicts since startup at 2005-11-18T12:13:38) 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort object: Surrogate oid= 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: at 1190763820> 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort object: Surrogate oid= 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: at 1190774252> all of which appear to be the result of the the single conflict error. The problems occur when the site is under load. To date I have not been able to find a simple instance that can reproduce the problem. Debugging in situ with a live system is not simple. In one instance, we have observed a loop where the conflict error generated resolution created another conflict error -- or so it appeared. On Sun, 13 Nov 2005, Peter Bengtsson wrote: > Have you considered upgrading to PostgreSQL? Things might have changed > since the last time I tried mysql but I'm pretty sure Postgresql and > psycopg deals much better with transactions in Zope. Admittedly, this > doesn't explain why you're getting into trouble just because you've > upgraded to zodb 3.4.2 > > On 11/13/05, Dennis Allison <[EMAIL PROTECTED]> wrote: > > Zope 2.8.4 > > ZEO 3.4.2 > > ZODB 3.4.2 > > Python 2.4.2 or 2.3.5 > > MySQL 4.0.20 > > MySQL-Python 1.2.1c3 (although we see the same problems with 1.2.0) > > MYSQLDA 2.0.9 > > > > We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but > > the ability to avoid read conflicts under ZODB3.4.2. We have been having > > a lot of problems: more conflict errors, release of unreleased locks in > > MySQL transactions, loss of session variables, and others. > > > > We use session variables and MySQL very heavily. Our MySQL tables are > > MyISAM and are not transactional. > > > > ZODB3 3.4.1 introduce some changes to the conflict resolution which > > appears to be a logical improvement but also appears to have acerbated > > our problems. In the ZODB# 3.4.2 system, if a commit fails, an exception > > is raised and the transaction is aborted. Zope then attempts to reschedule > > the transaction. The abort needs to roll back MySQL transactions that are > > part of the conflicting ZODB and the Session Variable Store so the > > restart is clean. > > > > In our case, we have been using MyISAM tables, which are not > > transactional. The rollbacks fail and the locking in the database adapter > > gets confused. Changing the table type from MyISAM to Innodb, which is > > transactional, should resolve the MySQL problems, although it has yet to > > be tested. > > > > Conflict errors also arise through the session variables. Since we have a > > multi-Zope system configuration sharing a single ZEO, the session > > variables are stored in a Temporary Storage object on the ZEO host. When > > a conflict occurs, a number of things, none good, apparently can happen. > > Session variable transactions cannot be rolled back (as the temporary > > storage mechanism is not transactional). And, at a different level, > > session variable persistence requires that transactions require a > > copy-out/copy-in mechanism to identify those changes which have been > > changed and need to persisted; an aborted transaction may not be marked as > > persistent and so will disappear at some apparently random time in the > > future when the temporary store is packed and cruft removed. > > > > Some refactoring of the programming can help minimize the potential for > > session variable conflicts, but they cannot be eliminated. > > > > If systems stability in the face of session variable conflicts is > > important, > > > > 1. Does the store for session variable need to be transactional > > and support rollbacks? > > > > and > > > > 2. What Temporary Storage object provides the right mechanism to > > manage session variables with rollback efficiently? > > > > > > > > ___ > > Zope maillist - Zope@zope.org > > http://mail.zope.org/mailman/listinfo/zope > > ** No cross posts or HTML encoding! ** > > (Related lists - > > http://mail.zope.org/mailman/listinfo/zope-announce > > http://mail.zope.org/mailman/listinfo/zope-dev ) > > > > > -- > Peter Bengtsson, > work www.fry-it.com > home www.peterbe.com > hobby www.issuetrackerproduct.com > -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/
Re: [Zope] session variables in the presence of conflicts
On Sun, 2005-11-13 at 01:39 -0800, Dennis Allison wrote: > Zope 2.8.4 > ZEO 3.4.2 > ZODB 3.4.2 > Python 2.4.2 or 2.3.5 > MySQL 4.0.20 > MySQL-Python 1.2.0 > MYSQLDA 2.0.9 > > We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but > the ability to avoid read conflicts under ZODB 3.4.2. We have been having > a lot of problems: more conflict errors, release of unreleased locks in > MySQL transactions, loss of session variables, and others. That sounds unfortunate. > We use session variables and MySQL very heavily. Our MySQL tables are > MyISAM and are not transactional. That also sounds unfortunate. ;-) > ZODB3 3.4.1 introduce some changes to the conflict resolution which ^ 3.4.2 I think you mean. > appears to be a logical improvement but also appears to have acerbated > our problems. In the ZODB# 3.4.2 system, if a commit fails, an exception > is raised and the transaction is aborted. Zope then attempts to reschedule > the transaction. The abort needs to roll back MySQL transactions that are > part of the conflicting ZODB and the Session Variable Store so the > restart is clean. Yes. Of course with MyISAM tables, this not really possible as you note below. > In our case, we have been using MyISAM tables, which are not > transactional. The rollbacks fail and the locking in the database adapter > gets confused. Changing the table type from MyISAM to Innodb, which is > transactional, should resolve the MySQL problems, although it has yet to > be tested. FWIW, InnoDB tables are a good idea in any case. But I'm suprised that a rollback which includes MyISAM tables actually causes failure. In my experience with the most recent version of Andy's MySQL database adapter, rollbacks don't actually do anything but they don't cause any failures either. This is also with a fairly recent MySQL (4.1.something). > Conflict errors also arise through the session variables. Since we have a > multi-Zope system configuration sharing a single ZEO, the session > variables are stored in a Temporary Storage object on the ZEO host. When > a conflict occurs, a number of things, none good, apparently can happen. > Session variable transactions cannot be rolled back (as the temporary > storage mechanism is not transactional). That is actually not true. The tempstorage mechanism is transactional, meaning that changes to a tempstorage will be committed or thrown out at transaction boundaries depending on the result of the participating data manager vote which determines whether a transaction should be committed or aborted. It is not undoable like FileStorage is, meaning you can't go back and selectively choose states to roll back to past the current database state, but it is indeed transactional. > And, at a different level, > session variable persistence requires that transactions require a > copy-out/copy-in mechanism to identify those changes which have been > changed and need to persisted; an aborted transaction may not be marked as > persistent and so will disappear at some apparently random time in the > future when the temporary store is packed and cruft removed. The temporary storage does not need to be packed unless there are cycles in the data. It is a "packless" storage in this regard. However, the sessioning machinery does do its own garbage collection of old data, which removes old data from the storage. But this machinery operates (logically) independently of whatever storage implementation you happen to be using to keep your session data manager. If you see session data disappear "randomly", this is a bug. However, if it can be explained by the inability of a particular transaction to commit due to bugs in participating data managers, it's not. > Some refactoring of the programming can help minimize the potential for > session variable conflicts, but they cannot be eliminated. > > If systems stability in the face of session variable conflicts is > important, > > 1. Does the store for session variable need to be transactional > and support rollbacks? It already is. > > and > > 2. What Temporary Storage object provides the right mechanism to > manage session variables with rollback efficiently? There is only one, and it is coded in such a way that it should provide this. You might need to provide some additional information about what kinds of errors you're experiencing; optimally you could provide a reproducible test case but experience has shown that with sessioning this is not always easy. Before doing much more investigation on the Zope end, you may want to get yourself to a place where you have recent versions of MySQL software and Andy's DA. - C ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev
Re: [Zope] session variables in the presence of conflicts
Have you considered upgrading to PostgreSQL? Things might have changed since the last time I tried mysql but I'm pretty sure Postgresql and psycopg deals much better with transactions in Zope. Admittedly, this doesn't explain why you're getting into trouble just because you've upgraded to zodb 3.4.2 On 11/13/05, Dennis Allison <[EMAIL PROTECTED]> wrote: > Zope 2.8.4 > ZEO 3.4.2 > ZODB 3.4.2 > Python 2.4.2 or 2.3.5 > MySQL 4.0.20 > MySQL-Python 1.2.0 > MYSQLDA 2.0.9 > > We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but > the ability to avoid read conflicts under ZODB3.4.2. We have been having > a lot of problems: more conflict errors, release of unreleased locks in > MySQL transactions, loss of session variables, and others. > > We use session variables and MySQL very heavily. Our MySQL tables are > MyISAM and are not transactional. > > ZODB3 3.4.1 introduce some changes to the conflict resolution which > appears to be a logical improvement but also appears to have acerbated > our problems. In the ZODB# 3.4.2 system, if a commit fails, an exception > is raised and the transaction is aborted. Zope then attempts to reschedule > the transaction. The abort needs to roll back MySQL transactions that are > part of the conflicting ZODB and the Session Variable Store so the > restart is clean. > > In our case, we have been using MyISAM tables, which are not > transactional. The rollbacks fail and the locking in the database adapter > gets confused. Changing the table type from MyISAM to Innodb, which is > transactional, should resolve the MySQL problems, although it has yet to > be tested. > > Conflict errors also arise through the session variables. Since we have a > multi-Zope system configuration sharing a single ZEO, the session > variables are stored in a Temporary Storage object on the ZEO host. When > a conflict occurs, a number of things, none good, apparently can happen. > Session variable transactions cannot be rolled back (as the temporary > storage mechanism is not transactional). And, at a different level, > session variable persistence requires that transactions require a > copy-out/copy-in mechanism to identify those changes which have been > changed and need to persisted; an aborted transaction may not be marked as > persistent and so will disappear at some apparently random time in the > future when the temporary store is packed and cruft removed. > > Some refactoring of the programming can help minimize the potential for > session variable conflicts, but they cannot be eliminated. > > If systems stability in the face of session variable conflicts is > important, > > 1. Does the store for session variable need to be transactional > and support rollbacks? > > and > > 2. What Temporary Storage object provides the right mechanism to > manage session variables with rollback efficiently? > > > > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) > -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] session variables in the presence of conflicts
Zope 2.8.4 ZEO 3.4.2 ZODB 3.4.2 Python 2.4.2 or 2.3.5 MySQL 4.0.20 MySQL-Python 1.2.0 MYSQLDA 2.0.9 We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but the ability to avoid read conflicts under ZODBÂ 3.4.2. We have been having a lot of problems: more conflict errors, release of unreleased locks in MySQL transactions, loss of session variables, and others. We use session variables and MySQL very heavily. Our MySQL tables are MyISAM and are not transactional. ZODB3 3.4.1 introduce some changes to the conflict resolution which appears to be a logical improvement but also appears to have acerbated our problems. In the ZODB# 3.4.2 system, if a commit fails, an exception is raised and the transaction is aborted. Zope then attempts to reschedule the transaction. The abort needs to roll back MySQL transactions that are part of the conflicting ZODB and the Session Variable Store so the restart is clean. In our case, we have been using MyISAM tables, which are not transactional. The rollbacks fail and the locking in the database adapter gets confused. Changing the table type from MyISAM to Innodb, which is transactional, should resolve the MySQL problems, although it has yet to be tested. Conflict errors also arise through the session variables. Since we have a multi-Zope system configuration sharing a single ZEO, the session variables are stored in a Temporary Storage object on the ZEO host. When a conflict occurs, a number of things, none good, apparently can happen. Session variable transactions cannot be rolled back (as the temporary storage mechanism is not transactional). And, at a different level, session variable persistence requires that transactions require a copy-out/copy-in mechanism to identify those changes which have been changed and need to persisted; an aborted transaction may not be marked as persistent and so will disappear at some apparently random time in the future when the temporary store is packed and cruft removed. Some refactoring of the programming can help minimize the potential for session variable conflicts, but they cannot be eliminated. If systems stability in the face of session variable conflicts is important, 1. Does the store for session variable need to be transactional and support rollbacks? and 2. What Temporary Storage object provides the right mechanism to manage session variables with rollback efficiently? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )