Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:  fixed
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 Note: This also broke MySQL but the tests never got this far since the
 Postgres builds blocked the queue.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:  fixed
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by apollo13):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 So the tests did work for me cause I already had the fix which github
 fails to display in the history *rage*
 
https://github.com/django/django/commit/56e54727661bc34bd2b6f9fa6a75f5370149256e
 and
 
https://github.com/django/django/commit/bacb097ac31af2479c43934747b34fad7c91f55c

 Did I already say that this is absolutely githubs fault? :þ

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 That's at least here not the case:
 {{{
 Installed 0 object(s) from 0 fixture(s)
 test_test_test (regressiontests.servers.tests.LiveServerAddress) ... ok
 test_database_writes (regressiontests.servers.tests.LiveServerDatabase)
 ... ok
 test_fixtures_loaded (regressiontests.servers.tests.LiveServerDatabase)
 ... ok
 test_404 (regressiontests.servers.tests.LiveServerViews) ... ok
 test_environ (regressiontests.servers.tests.LiveServerViews) ... ok
 test_media_files (regressiontests.servers.tests.LiveServerViews) ... ok
 test_static_files (regressiontests.servers.tests.LiveServerViews) ... ok
 test_view (regressiontests.servers.tests.LiveServerViews) ... ok

 --
 Ran 8 tests in 2.356s
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 `LiveServerTestCase` needs `request_finished` to close connections;
 disabling it globally for tests was wrong.

 I'm attaching a patch that's supposed to fix the problem...

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by aaugustin):

 * status:  closed => new
 * resolution:  fixed =>


Comment:

 My fix makes `servers.LiveServerDatabase.test_fixtures_loaded` hang under
 PostgreSQL.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:  fixed
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by Aymeric Augustin ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"fd1279a44df3b9a837453cd79fd0fbcf81bae39d"]:
 {{{
 #!CommitTicketReference repository=""
 revision="fd1279a44df3b9a837453cd79fd0fbcf81bae39d"
 [1.5.x] Fixed #19519 -- Fired request_finished in the WSGI iterable's
 close().

 Backport of acc5396.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:  fixed
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by Aymeric Augustin ):

 In [changeset:"acc5396e6d0ac49ae9dc6abc08903b81e6553199"]:
 {{{
 #!CommitTicketReference repository=""
 revision="acc5396e6d0ac49ae9dc6abc08903b81e6553199"
 Fixed #19519 -- Fired request_finished in the WSGI iterable's close().
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 -1 on a new signal, I'd say ship it as is.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 If we want to avoid changing the semantic of request_finished we can also
 introduce a new signal. We'd have to find a not-too-confusing name.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 Summary of an IRC discussion:

 - We believe that current implementations of this idea (eg. gunicorn's
 async workers) use green threads that honor Python's threading API; the
 database connection is a green thread local.
 - If that weren't true, Django would break horribly anyway, independently
 of streaming responses.
 - The way `gevent` solves blocking I/O makes the PEP's point moot; it
 makes the write() API just as "pseudo-non-blocking" as the iterator API.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 The real question is: does any WSGI server actually used to serve real
 websites sport this behavior?

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by akaariai):

 I am wondering if pushing the request_finished signal to late stages is
 actually safe to do. Question no. 2 from pep 
 [http://www.python.org/dev/peps/pep-/#questions-and-answers] says
 that:
 {{{
 If we supported only the iteration approach, then current frameworks
 that assume the availability of "push" suffer. But, if we only support
 pushing via write(), then server performance suffers for transmission
 of e.g. large files (if a worker thread can't begin work on a new
 request until all of the output has been sent). Thus, this compromise
 allows an application framework to support both approaches, as
 appropriate, but with only a little more burden to the server
 implementor than a push-only approach would require.
 }}}

 To me this suggest it is possible a different thread will finish the
 request and/or there is another request processed at the same time by the
 same thread. This would mean that we could be closing connections for
 different threads or different requests. I hope I am 100% wrong here.

 If the above is the case, this is going to get a lot more complex. The
 connection handling is bound to threads, not requests. What we could do is
 detach the connection from the thread at the old request_finished point,
 and then when the request is actually finished (the new request_finished
 point) we could close the connection.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Ready for
 Keywords:  connection,  |  checkin
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by aaugustin):

 * needs_tests:  1 => 0
 * stage:  Accepted => Ready for checkin


Comment:

 This isn't testable within Django's test framework because the test client
 explicitly disables closing the database connection at the end of each
 request (since #8138).

 I've used Florian's test project to validate manually that the pull
 request fixes the issue.

 Per PEP , "the close() method requirement is to support resource
 release by the application". Currently, Django uses the `request_finished`
 signal to close the connections to the databases and caches. That's
 exactly the purpose of `close()`, and therefore I think it's a good idea
 to send the signal from there.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  1|
  Needs tests:  1|
Easy pickings:  0|
-+-
Changes (by aaugustin):

 * status:  new => assigned
 * owner:  nobody => aaugustin
 * has_patch:  0 => 1
 * needs_tests:  0 => 1


Comment:

 https://github.com/django/django/pull/612

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 Replying to [comment:9 apollo13]:
 > Related: http://bugs.python.org/issue16220 -- We either have to raise
 the required python versions or fix it in our wsgiref subclass (probably
 the later since Python didn't fix it in 2.6, unless it wasn't broken which
 I doubt).

 This fix will be available in Python 2.7.4, which isn't released yet.

 If we want to fix this by triggering `request_finished` in the WSGI
 iterable's `close()` method we must ensure it's reliably closed.

 I've re-opened #16241.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 Replying to [comment:6 akaariai]:
 > Do we have a problem with transactions here? The transaction used for
 the request could be closed before the request is actually generated.

 This is less likely to bite people in real life. There are obvious uses
 cases for reading from the database while generating a streaming response
 (eg. huge CSV exports); I don't see writing to the database as being as
 common.

 It's tracked by #5241, and I don't think it is a release blocker.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 Related: http://bugs.python.org/issue16220 -- We either have to raise the
 required python versions or fix it in our wsgiref subclass (probably the
 later since Python didn't fix it in 2.6, unless it wasn't broken which I
 doubt).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 Ok, so I attached an initial patch. Anssi provided some valuable feedback:
 He suggested that we only do the .close dance if we are actually working
 with a StreamingResponse to keep the patch backwards-compatible (although
 we don't know if it is backwards-incompatible yet). I somewhat agree with
 that, but on the other hand I'd like to handle both cases in the same way.

 Next question is how to handle streaming responses in the test client.
 Currently they are not consumed at all and Anssi suggested to consume them
 always (probably add a flag later on to prevent consuming them, but that's
 probably not needed for 1.5).

 Last thing: I am still thinking how to test that since the TestClient uses
 it's own WSGI handler which does stuff a bit different (it actually
 disconnects the close_connections receiver before it fires
 request_finished ;)).

 Any thoughts from someone with more knowledge of the WSGI protocol than me
 would be appreciated :)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 Yes, the transaction middleware will also commit the transaction before
 the response is generated. Same goes for decorators like
 commit_on_success. But independent of that, the whole connection is closed
 before the response is generated.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by akaariai):

 Do we have a problem with transactions here? The transaction used for the
 request could be closed before the request is actually generated. If this
 is the case, then documenting this with "if using transaction middleware
 the transaction used for the request will be closed before the request is
 generated" seems like the easy way out. Another option is to add hooks for
 request finished with success/exception, but this is out of scope for 1.5.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 The request_finished signal should be hooked on the close() method of the
 WSGI iterable.

 As pointed out in comment 4 this was already broken in the old-style
 streaming responses.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by amosonn@…):

 It is probably best to make sure this works for iterators passed to
 regular HttpResponse objects as well, even though it is deprecated.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Release blocker  | Triage Stage:  Accepted
 Keywords:  connection,  |  Needs documentation:  0
  managment, closing, mysql, |  Patch needs improvement:  0
  innodb, signal, generator  |UI/UX:  0
Has patch:  0|
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by apollo13):

 * cc: apollo13 (added)
 * version:  1.4 => master
 * severity:  Normal => Release blocker
 * stage:  Unreviewed => Accepted


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:
 Keywords:  connection,  |  Unreviewed
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  0|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by apollo13):

 Raising this issue to "release blocker" since streaming responses are new
 in 1.5 and somewhat broken in that regard! I attached a project which
 demonstrates the issue, see the following server log:
 {{{
 
 
 () {'signal': ,
 'sender': }
 
 [29/Dec/2012 03:54:12] "GET /testing/ HTTP/1.1" 200 15


 
 
 () {'signal': ,
 'sender': }
 
 None
 request already finished: True
 [29/Dec/2012 03:54:16] "GET /testing_generator/ HTTP/1.1" 200 18
 }}}

 What does this mean? When using StreamingHttpResponse the request_finished
 signal is sent before the response is actually generated, meaning database
 connections are already closed and a new query inside the iterator would
 cause a new connection to open.

 A fix might be something along the lines of
 http://werkzeug.pocoo.org/docs/wsgi/#werkzeug.wsgi.ClosingIterator -- eg
 returning the response and issuing the request_finished signal after the
 upstream wsgi gateway exhausted the iterator.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #19519: Django fails to close db connections at end of request-response cycle

[Django]

#19519: Django fails to close db connections at end of request-response cycle
-+-
 Reporter:  amosonn@…|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:
 Keywords:  connection,  |  Unreviewed
  managment, closing, mysql, |  Needs documentation:  0
  innodb, signal, generator  |  Patch needs improvement:  0
Has patch:  0|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by aaugustin):

 * needs_better_patch:   => 0
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 I remember making some fixes in this area when I introduced explicit
 support for streaming responses. If this is indeed related to streaming
 responses it's most likely fixed in master.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.