Hello Aymeric, regarding django.setup() idempotence I continued discussion in the PR.
Regarding django.setup() tweakability, the generic need is simply to have a hook, somewhere for users to put early init code, regardless of the way django is launched. Code that needs to execute this early are mostly (if not all) monkey-patching the environnement, a little like a LD_PRELOAD logic python-style. This can be: - for (of course) the init of django-compat-patcher - greening of the environment (e.g. as https://github.com/erickponce/django-db-geventpool recommends, for Gevent or Eventlet) - early tweaking of some process aspects likes socket timeouts (they broke for me when used along with mutiprocessing, but several years ago) - possibly setup of specific test scaffolding, for which late mock()-style things aren't enough (I have no precise scenario in mind for this one, even though I know we did suffer with a Jpype bridge to a java LDAP client, which required tons of care to not break, including being initialized rather early along with process signal handlers) Without a hook built into django, users that need such early setup have to duplicate their efforts for each way Django can be launched, and they must call their patching code in manage.py, in the wsgi file, in their custom scripts, and in their specific runners likes pytest-django (this last case is quite uneasy to handle at the moment, due to its very early setup of Django). I'm afraid I can't provide much code to help the pondering; simply, without builtin hook, the user has to battle to ensure the kind of lines like the following are always executed before django.setup(). If Django provides a hook, it's so much less to worry about. from psycogreen.gevent import patch_psycopg patch_psycopg() There are surely lots of ways to provide such a hook, via new settings, or new environment variables, or modules following precise naming conventions that django would systematically try to import... I have no idea how exactly my approach is rated amongst them. As for alternatives to these early setup hooks, I'm alas clueless... regards, Pakal On Sunday, June 30, 2019 at 10:25:15 AM UTC+2, Aymeric Augustin wrote: > > Hello Pakal, > > *Making django.setup() idempotent* > > In https://code.djangoproject.com/ticket/28752, you say: > > > *> I've been bitten numerous times by the impredictable behaviour of > django when django.setup() was called numerous times.> In the old days I > had exceptions, now it's mainly subtle breakages of logging configuration.* > > This is fuzzy for a bug report. Ideally, you should say: > > - What you're doing (minimal reproduction instructions, within supported > use of Django) > - What results you're expecting > - What results you're getting > > Anyway it suggests that the problem is about logging. In this comment > <https://github.com/django/django/pull/11440#pullrequestreview-250231882> I > said that the only possible bug is that `configure_logging()` isn't > idempotent. Your answer > <https://github.com/django/django/pull/11440#issuecomment-504116261> confirms > that the issue is really about logging. Let's fix configure_logging(), then! > > Barring new arguments in favor of the proposed approach, which involves > nested locks, I'm -1. Nested locks create a possibility of deadlocks. See > my comment on the PR for details. > > *Making django.setup() tweakable* > > Again, the rationale is fuzzy. I'm seeing two use cases: > > - pytest-django > - django-compat-patchers > > One more or one less monkey patch isn't going to make a difference to > these projects ;-) > > I'm ready to accept that there's a need for projects that aren't big hacks > but I need to see it explained clearly and specifically. I'd really like to > see a feature request in the form of: > > - I want to do X, which is a reasonable thing to do and stays within > supported use of Django, except I need to override django.setup() > - Here's what the code looks like with a workaround (monkey-patching, > probably) > - Here's what the code would look like with the improvement I'm asking for > (hopefully much better) > - Here are the alternatives I considered and why I rejected them > - Other use cases also include Y and Z > > I'm asking this because concrete use cases will put us in a better > position to find the best solution. > > Best regards, > > -- > Aymeric. > > > > On 27 Jun 2019, at 23:26, Pkl <chambo...@gmail.com <javascript:>> wrote: > > Hello everyone, > > I'm bringing here, for broader review and discussion, the subject of > making django setup more "tweakable": > https://code.djangoproject.com/ticket/30536 > https://github.com/django/django/pull/11435 > > There is indeed a need for doing pre/post operations at that crucial > moment of the framework lifetime, especially to apply compatibility shims > (greening-related, API stability-related...) and setup test scaffolding, > stubs etc. > > A separate PR has been created to handle threadsafety/idempotence ( > https://github.com/django/django/pull/11440) when multiple actors try to > setup django concurrently, but a discussion remains on how to potentially > allow users to specify an alternative setup > > I originally did a POC with a new django setting, but if we want to have > full control, even settings might not be normally loaded at that stage. So > maybe, as apollo13 suggested, an environment variable, like > DJANGO_SETUP_CALLABLE, complementing the DJANGO_SETTINGS_MODULE one? > What are your thoughts about this ? > > thanks, > regards, > Pakal > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-d...@googlegroups.com <javascript:>. > To post to this group, send email to django-d...@googlegroups.com > <javascript:>. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/0206c98c-199f-49f9-9ff7-cba736e71224%40googlegroups.com > > <https://groups.google.com/d/msgid/django-developers/0206c98c-199f-49f9-9ff7-cba736e71224%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a1db198b-27ba-4988-a9ad-67a2cf23e32e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
