Hi Tore, I’ve been meaning to check whether your use case was supported, because it should be. I’m sorry, it took me a few days to find the time to investigate.
While I was testing, I produced an exception which looks similar to the problem you originally reported: https://code.djangoproject.com/ticket/26340. Specifically, could you try adding `self.needs_rollback = False` at the bottom of the `BaseDatabaseWrapper.rollback()` and tell me if that helps? (Since I don’t remember hearing you mention savepoints, you can probably ignore everything I say about savepoints.) > On 07 Mar 2016, at 12:55, Tore Lundqvist <t...@mima.x.se> wrote: > > an example of when it happens is when: > > Starting with auto commit on. > transaction.set_autocommit(False) > ... a lot of time passes without the connection being used so it times out.... > close_old_connections() There’s a long discussion of this use case here: https://code.djangoproject.com/ticket/21597. > Now autocommit is on again. I'm not saying that it's a bug, but it's > inconvenient. This surprises me. The connection shouldn’t reopen at all. It should become unusable, for the reasons explained in that issue (ticket 21597). Regardless, at any point in the code where you expect not to use the database connection for some time, you should close it explicitly with connection.close(). Then you can call set_autocommit(False) when you’re about to start using the database connection again. Since I don’t expect the database timeout to be reached while handling a HTTP request, I assume this happens in a long-running management command. You could have another settings module just for the purpose of running that command. It would be the same as your regular settings module, except it would set the AUTOCOMMIT option to False in the database configuration. In that case, you wouldn’t need to call set_autocommit(False). > The problem is that the legacy code uses COMMIT side effects to manage disk > writes and avoid deadlocks, securing an adequate transactional integrity is > often not an issue. That’s an interesting way to use transactions, and to be honest, one I never imagined ;-) While I wouldn’t recommend that design for new code, Django shouldn’t prevent you from continuing to use it. Obviously, some other users of Django need transactional integrity. We can’t relax that constraint when discussing changes to Django. > So I'm trying to disable or at least use as little as possible of Djangos > transaction management for this code. After transaction.set_autocommit(False), transaction.commit() and transaction.rollback() seem to work fine. As I pointed out in my previous messages, you have to rollback with transaction.atomic() when you’re catching IntegrityError. I thought you could rollback with transaction.rollback() or transaction.savepoint_rollback(). It turns out you also need to call transaction.set_rollback(False). Essentially this tells Django “hey, I’m happy to continue from this point, just go ahead”. I think Django could do better. That’s why I filed the issue I mentioned at the beginning of this email (ticket 26340). Given MySQL’s interesting approach to transactional integrity, you can call transaction.set_rollback(False) after a query that failed with an IntegrityError and keep going. In case that’s useful, here are some patterns that work currently. A and B are two distinct sessions. I’m using session B to check whether the changes from session A have been committed. I have a user model with a unique email field. I used PostgreSQL because I don’t have a MySQL instance around. If you’re getting different results on MySQL, it’s a bug. A >>> from django.db import IntegrityError, transaction A >>> transaction.set_autocommit(False) A >>> User.objects.create(email='us...@example.com') A <User: us...@example.com> A >>> User.objects.create(email='us...@example.com') A <User: us...@example.com> B >>> User.objects.filter(email__startswith='user') B [] A >>> transaction.commit() B >>> User.objects.filter(email__startswith='user') B [<User: us...@example.com>, <User: us...@example.com>] A >>> try: A ... with transaction.atomic(): A ... User.objects.create(email='us...@example.com') A ... except IntegrityError: A ... print("duplicate") A ... A <User: us...@example.com> A >>> try: A ... with transaction.atomic(): A ... User.objects.create(email='us...@example.com') A ... except IntegrityError: A ... print("duplicate") A ... A duplicate A >>> try: A ... with transaction.atomic(): A ... User.objects.create(email='us...@example.com') A ... except IntegrityError: A ... print("duplicate") A ... A <User: us...@example.com> B >>> User.objects.filter(email__startswith='user') B [<User: us...@example.com>, <User: us...@example.com>] A >>> transaction.commit() B >>> User.objects.filter(email__startswith='user') B [<User: us...@example.com>, <User: us...@example.com>, <User: us...@example.com>, <User: us...@example.com>] A >>> User.objects.create(email='us...@example.com') A <User: us...@example.com> A >>> User.objects.create(email='us...@example.com') A <User: us...@example.com> B >>> User.objects.filter(email__startswith='user') B [<User: us...@example.com>, <User: us...@example.com>, <User: us...@example.com>, <User: us...@example.com>] A >>> transaction.rollback() B >>> User.objects.filter(email__startswith='user') B [<User: us...@example.com>, <User: us...@example.com>, <User: us...@example.com>, <User: us...@example.com>] Best regards, -- Aymeric. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/EC2ECCC9-02EB-408D-B6B1-E21BB029DB22%40polytechnique.org. For more options, visit https://groups.google.com/d/optout.
