Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Old description:

> (I posted this on the ML initially, but I suspect it's a bug in the
> testing framework. A complete example is attached. Empty unit test, with
> no routers and a read replica DB setup gives a
> TransactionManagementError)
>
> --
>
> I started assessing multi-db support in Django 1.5.1, I'm interested in
> having read replicas and a write master.
>
> I've figured out all the easy stuff: using TEST_MIRROR in the DATABASES
> setting, writing a simple router that splits read and write traffic, all
> according to the documentation. I can run a test server and issue some
> calls manually, but I have not been able to run simple unit tests.
>
> The most simple test I can build fails during _pre_setup() with a
> "TransactionManagementError: Transaction managed block ended with pending
> COMMIT/ROLLBACK"
>
> My databases are MySQL EC2 RDS instances, the replication is working
> fine, the tables are InnoDB etc.
>
> I know there is active development around transaction management code in
> Django lately, so maybe I've stumbled on a bug? I'm attaching a simple
> test setup that should make it easy to reproduce.
>
> Any advice would be greatly appreciated.
>
> Cheers,
> TTimo
>
> ./manage.py test readreplica
> Creating test database for alias 'default'...
> E
> ==
> ERROR: testTest (readreplica.tests.Test)
> --
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py",
> line 240, in __call__
> self._pre_setup()
>   File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py",
> line 462, in _pre_setup
> self._fixture_setup()
>   File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py",
> line 822, in _fixture_setup
> if not connections_support_transactions():
>   File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py",
> line 809, in connections_support_transactions
> for conn in connections.all())
>   File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py",
> line 809, in 
> for conn in connections.all())
>   File "/usr/local/lib/python2.7/dist-
> packages/django/utils/functional.py", line 48, in __get__
> res = instance.__dict__[self.func.__name__] = self.func(instance)
>   File "/usr/local/lib/python2.7/dist-
> packages/django/db/backends/__init__.py", line 627, in
> supports_transactions
> self.connection.leave_transaction_management()
>   File "/usr/local/lib/python2.7/dist-
> packages/django/db/backends/__init__.py", line 319, in
> leave_transaction_management
> "Transaction managed block ended with pending COMMIT/ROLLBACK")
> TransactionManagementError: Transaction managed block ended with pending
> COMMIT/ROLLBACK
>
> --
> Ran 0 tests in 0.084s
>
> FAILED (errors=1)
> Destroying test database for alias 'default'...

New description:

 (I posted this on the ML initially, but I suspect it's a bug in the
 testing framework. A complete example is attached. Empty unit test, with
 no routers and a read replica DB setup gives a TransactionManagementError)

 --

 I started assessing multi-db support in Django 1.5.1, I'm interested in
 having read replicas and a write master.

 I've figured out all the easy stuff: using TEST_MIRROR in the DATABASES
 setting, writing a simple router that splits read and write traffic, all
 according to the documentation. I can run a test server and issue some
 calls manually, but I have not been able to run simple unit tests.

 The most simple test I can build fails during _pre_setup() with a
 "TransactionManagementError: Transaction managed block ended with pending
 COMMIT/ROLLBACK"

 My databases are MySQL EC2 RDS instances, the replication is working fine,
 the tables are InnoDB etc.

 I know there is active development around transaction management code in
 Django lately, so maybe I've stumbled on a bug? I'm attaching a simple
 test setup that should make it easy to reproduce.

 Any advice would be greatly appreciated.

 Cheers,
 TTimo

 {{{
 ./manage.py test readrep

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by aaugustin):

 * status:  new => closed
 * resolution:   => needsinfo


Comment:

 After changing the settings file to:

 {{{
 SECRET_KEY='oh hay this is a test'

 DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.mysql',
 'NAME': 'django_test',
 'USER': 'root',
 },
 'slave': {
 'ENGINE': 'django.db.backends.mysql',
 'NAME': 'django_other',
 'USER': 'root',
 # https://docs.djangoproject.com/en/dev/topics/testing/advanced
 /#topics-testing-masterslave
 'TEST_MIRROR' : 'default',
 },
 }

 INSTALLED_APPS = ['readreplica']
 }}}

 The example works for me on Django 1.7:

 {{{
 (django-dev)myk@mYk readreplica % ./manage.py test --settings=settings
 ~/Documents/dev/django/readreplica
 Creating test database for alias 'default'...
 testTest
 .
 --
 Ran 1 test in 0.054s

 OK
 Destroying test database for alias 'default'...
 }}}

 Since this was reopened anonymously there's nothing we can do about it.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.981f5bb045c5c79b941ba144da9c8b9c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by cad106uk):

 I had this problem this morning. For me it turned out not to be a problem
 with my django install. It turned out to be a problem with my .sqlite3
 file. When I deleted this file and ran the tests again the problem went
 away.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.262d7ec4c76ce27f6e29fd3645d2c352%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by anonymous):

 Also this captcha like math equation for spam often fails here if the
 result is supposed to be a negative number.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.6cb63929b7d8e6d3312103ffd6c74645%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by anonymous):

 * status:  closed => new
 * resolution:  needsinfo =>


Comment:

 I've experienced the same bug on both django 1.5 and django 1.6  I too am
 using multi-db read replica with TEST_MIRROR set.  If I remove the slave
 and the db router the test runs fine.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.1c072345ff15309d61f900b088e5ffc7%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  closed
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:  needsinfo
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by timo):

 * status:  new => closed
 * resolution:   => needsinfo


Comment:

 As noted above, please re-open if this is still an issue with Django 1.6.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.2ef89cc47b001212dc5d07010276c5fd%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by aaugustin):

 This cannot be a side effect of the recent work on transaction management,
 since that work didn't touch the stable/1.5.x branch.

 The previous transaction management was so buggy that it was entirely
 removed and replaced. That could explain why there's little interest in
 fixing it...

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.53c1065612a4a5ed67e6e44aa9848a9a%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #20373: Simple Multi-DB read replica unit test raises TransactionManagementError

[Django]

#20373: Simple Multi-DB read replica unit test raises TransactionManagementError
---+--
 Reporter:  TTimo  |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  Testing framework  |  Version:  1.5
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by mwaterfall):

 * needs_better_patch:   => 0
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 I'm also experiencing this issue. The ticket has been opened for 7 weeks
 without a reply which is a bit concerning :S

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.77309f871a3c341533a933f5e0b48be4%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.