Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:  fixed
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by Anssi Kääriäinen ):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [f572ee0c65ec5eac9edb0cb3e35c96ec86d89ffb]:
 {{{
 #!CommitTicketReference repository=""
 revision="f572ee0c65ec5eac9edb0cb3e35c96ec86d89ffb"
 Fixed #16047 -- Restore autocommit state correctly on psycopg2

 When the postgresql_psycopg2 backend was used with DB-level autocommit
 mode enabled, after entering transaction management and then leaving
 it, the isolation level was never set back to autocommit mode.

 Thanks brodie for report and working on this issue.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by kmike):

 * cc: kmike84@… (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by brodie):

 OK, I agree that your fix is the right approach.

 I have a new patch series that incorporates your changes. The first patch
 fixes the `supports_transactions` DB feature being `False` when the test
 databases are using autocommit mode. The second patch is based on your
 patch, but I've moved database creation/deletion into setUp/tearDown
 methods and made more fine-grained tests in the test case. The entire test
 case is skipped if we aren't using psycopg2.

 I also ran into an issue with your hardcoded isolation level constants. I
 think they might've changed across psycopg2 releases, so to be safe, I
 import them from psycopg2.

 The patch series is based against trunk (at [17688]).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by akaariai):

 What happens is:
   - when you enter transaction management "True" is appended to the
 transaction_state stack.
   - when you leave transaction management, currently top of the stack
 (True) is given to _leave_transaction_management(). This tells "where we
 are coming from".
   - my patch changes this in a way that the "True" is popped from the
 stack, and then what is left is given to _leave_transaction_management.
 Now, this is correct, because when setting the isolation level, we are not
 interested if we came from a managed block or not, we are interested if we
 are going into a managed state.
   - ignoring totally the managed parameter is wrong, because you can stack
 enter/leave calls. This would lead into situation where the first leave
 always sets you into autocommit mode, no matter how many enter calls there
 have been before. This is tested in the patch.

 About the managed parameter: don't ask. The transaction management code
 could be cleaner...

 About the try-except: True, that change is not necessary, it just seemed
 better to me. "Pop the stack, it is an exception if it is empty".

 I ran the full test suite and got some errors. However, I don't think they
 are related to this patch, but to some other recent commits. If you can
 review & mark the patch as ready for committer that would be great.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by brodie):

 It's been a while since I've looked at this issue, but let me see if I've
 got the whole picture right:

 - Currently, autocommit isn't restored because the psycopg2 backend won't
 restore it unless it's already out of Django's "managed" mode. But managed
 mode is turned on when entering a transaction, so you get stuck at the
 higher isolation level.

 - My patch removes the "not managed" check from
 _leave_transaction_management(), so it no longer cares whether you're in
 managed mode or not -- it'll go back to autocommit regardless.

 - Your patch disables managed mode *before* calling
 _leave_transaction_management(), so managed will be False, and it will end
 up restoring autocommit.

 I'm not entirely sure, but it seems like your patch makes more sense. That
 said, I don't think the try/except stuff you've added is necessary -- you
 should just move the
 "self._leave_transaction_management(self.is_managed())" call so it's after
 the existing self.transaction_state handling code.

 Also, having looked at this code again, I don't think I fully understand
 the enter_transaction_mangement()'s managed parameter. It seems like if
 you're in autocommit mode and set it to False, you stay in autocommit
 mode. But after you've called that method, even if you set managed=False,
 self.is_managed() will still be True, and it's still going to try to
 restore autocommit mode in _leave_transaction_management() (which doesn't
 really matter, I guess).

 I guess my real question is this: how does that managed parameter relate
 to is_managed()? They don't seem to be the same thing.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by akaariai):

 * cc: anssi.kaariainen@… (added)
 * needs_better_patch:  1 => 0


Comment:

 I think I have nailed this one. The problem was that in
 _leave_transaction_management, the managed variable was telling if we were
 coming from a managed transaction. However, that is not interesting. The
 interesting thing is if we are going into still managed state or not.

 I also rewrote the tests to fix enter/leave management stacking, and to
 work even if the settings do not contain the autocommit flag. This is a
 must in my opinion, as running the test suite with autcommit = True isn't
 something done regularly.

 I left the supports_transactions fix out of this. However, why not just
 return True when using PostgreSQL, SQLite or Oracle?

 I haven't ran the full test suite, which in this case seems necessary. I
 would like to mark this ready for committer (assuming tests pass), but I
 guess I am not allowed to do that... Would be nice to get this into 1.4.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  1
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by IonelMaries):

 * cc: ionel.mc@… (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
 Reporter:  brodie   |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.3
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  psycopg2 autocommit  |  Needs documentation:  0
  transactions   |  Patch needs improvement:  1
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by dikbrouwer):

 * cc: dick@… (added)
 * ui_ux:   => 0


Comment:

 Hi brodie, any updates on this patch? It seems to be critical to anybody
 who is using manual transaction management and PGBouncer.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
   Reporter:  brodie |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  1.3|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:  psycopg2 autocommit
Needs documentation:  0  |  transactions
Patch needs improvement:  1  |  Has patch:  1
 |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by brodie):

 * needs_better_patch:  0 => 1


Comment:

 I'll attempt to revise the patch so the new tests don't blow up with other
 DBs.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
   Reporter:  brodie |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  1.3|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:  psycopg2 autocommit
Needs documentation:  0  |  transactions
Patch needs improvement:  0  |  Has patch:  1
 |Needs tests:  0
 |  Easy pickings:  0
-+-

Comment (by kgibula):

 Patch also applies to trunk and tests pass for PostgreSQL 8.4 and psycopg2
 2.4.1. On other backends, however, '!DatabaseWrapper' object doesn't have
 an attribute 'isolation_level' so tests result in attribute errors (I
 tried MySQL and SQLite).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
   Reporter:  brodie |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  1.3|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:  psycopg2 autocommit
Needs documentation:  0  |  transactions
Patch needs improvement:  0  |  Has patch:  1
 |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by aaugustin):

 * stage:  Unreviewed => Accepted


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #16047: postgresql_psycopg2 never restores autocommit mode when leaving transaction management

[Django]

#16047: postgresql_psycopg2 never restores autocommit mode when leaving 
transaction
management
-+-
   Reporter:  brodie |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  1.3|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage: |   Keywords:  psycopg2 autocommit
  Unreviewed |  transactions
Needs documentation:  0  |  Has patch:  1
Patch needs improvement:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by brodie):

 * needs_better_patch:   => 0
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 Also, I should mention that I've run the test suite with my patches
 applied against the releases/1.3.X branch. All tests pass with DB-level
 autocommit disabled and enabled. I'm using psycopg2 2.4.1 and PostgreSQL
 9.0.4. I haven't tested other combinations of psycopg2, PostgreSQL, or
 other DBs.

 Here are the two test settings files I used:

 {{{
 #!python
 DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'djangotest',
 'USER': 'djangotest',
 'PASSWORD': 'djangotest',
 },
 'other': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'djangotestother',
 'USER': 'djangotestother',
 'PASSWORD': 'djangotestother',
 }
 }
 }}}

 {{{
 #!python

 DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'djangotest',
 'USER': 'djangotest',
 'PASSWORD': 'djangotest',
 'OPTIONS': {'autocommit': True},
 },
 'other': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'djangotestother',
 'USER': 'djangotestother',
 'PASSWORD': 'djangotestother',
 'OPTIONS': {'autocommit': True},
 }
 }
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.