I'm not starting threads by hand but rather a pool which handles any threads for me, I basically just send a function to the pool and leave it run. You are right, I could wrap every function sent to the pool with the code you proposed, but I also don't want to open and close a connection on every function call, but instead only when the thread from the pool is no longer required and disposed, pretty much on application exist, although the pool handler can do what ever it wants with the threads.
El jueves, 2 de junio de 2016, 20:22:21 (UTC-3), Stephen Butler escribió: > > I'm still a bit confused. What is the advantage of having connections > closed automatically when the thread exits? It seems to me that you can > quickly solve your problem by modifying your thread start routines: > > from django.db import connection > from contextlib import closing > > def my_thread_start(): > with closing(connection): > # do normal work > > You can even create a quick decorator if that's too much modification. > > On Thu, Jun 2, 2016 at 5:48 PM, Cristiano Coelho <cristia...@gmail.com > <javascript:>> wrote: > >> Florian, >> >> Sorry about the SIGTERM and SIGKILL confusion, I think I read somewhere >> some time ago that SIGTERM would instantly finish any pending request, so I >> assumed it would also kill any thread in not a really nice way. However now >> that you mention it, there's one SIGKILL from the apache logs (compared to >> the thousands of sigterm due to restarts). However the connections that >> were somehow stuck and never close dated from about 2 weeks ago, yes, there >> were connections that were opened 2 weeks ago and never closed, even if >> apache was restarded many times every day! >> >> About thread pool, I'm talking about python's thread pool I'm using to >> offload work, not any django's pool, and these pools are the ones I have no >> control over its threads as they are completely managed by the thread pool >> library. >> >> 3 days has passed since I noticed those hanged out connections and the >> issue didn't repeat again yet, maybe it was some really odd condition that >> caused them, but the thread pool's threads connections are indeed being >> correctly closed on servers restart, so a very odd case created those >> hanged connections. >> >> So just to be sure, is SIGTERM actually propagated to python code so it >> can gracefully kill all threads, garbage collect and close connections? >> Would a SIGKILL actually prevent any kind of cleanup leaving a chance for >> python/django leave some connections opened? >> >> Maybe this is a postgres issue instead that happened for some very odd >> reason. >> >> >> Finally, would it be possible through any kind of callbacks of the thread >> local object to fire a connection close before a thread dies? This would >> certainly help rather than waiting for the connection to get garbage >> collected. You mentioned that connections could end up being shared by >> threads, but I don't see that being something being done in django at all. >> >> >> El jueves, 2 de junio de 2016, 19:32:15 (UTC-3), Florian Apolloner >> escribió: >>> >>> On Thursday, June 2, 2016 at 11:55:41 PM UTC+2, Cristiano Coelho wrote: >>>> >>>> Not always, for example, on amazon elastic beasntalk when you either >>>> restart the app server or upload a new version, it basically kills apache >>>> and all WSGI processes through a sigterm >>>> >>> >>> A SIGTERM is a normal signal an should cause a proper shutdown. >>> >>> >>>> so those thread pools are probably killed in a bad way and you don't >>>> really have control over that. >>>> >>> >>> Absolutely not, you are mixing up SIGTERM and SIGKILL. >>> >>> >>>> Also you don't really have control on the life of a thread pool thread, >>>> so a given thread could be gracefully stopped by the pool implementation >>>> >>> >>> Once again: there is no pool. >>> >>> As ayneric pointed out, it seems like those connections are correctly >>>> closed most of the time when a thread dies, but for some reason, postgres >>>> would keep some connections opened. >>>> >>> >>> If a connection is closed properly, postgres will close it accordingly. >>> The only way possible for a connection to stay open while the app is gone >>> is that you are running into tcp timeouts while getting killed with SIGTERM >>> (dunno if the postgres protocol has keep alive support on the protocol >>> level, most likely not). As long as you are not sending a SIGTERM, python >>> should clean up properly which should call garbage collection, which then >>> again should delete all connections and therefore close the connection. Any >>> other behavior seems like a bug in Python (or maybe Django, but as long as >>> Python shuts down properly I think we are fine). >>> >>> Are there any rare cases where even if the thread is stopped the >>>> connection won't be closed? The only thing I can think of are that those >>>> threads are never garbage collected or something. >>>> >>> >>> Depends on the python version you are using, especially thread local >>> behavior changed a lot… >>> >>> Cheers, >>> Florian >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-develop...@googlegroups.com <javascript:>. >> To post to this group, send email to django-d...@googlegroups.com >> <javascript:>. >> Visit this group at https://groups.google.com/group/django-developers. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/562bf2f9-8a44-495c-bacd-9a42012aa011%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-developers/562bf2f9-8a44-495c-bacd-9a42012aa011%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b54b3699-d755-4d19-ac9f-ee0c0fb69d13%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
