Re: Setting dictionaries (was Re: integrating django-secure)
It does slight more than that - settings configured and settings used are
two very different things at present - to use our canonical example
MIDDLEWARE_CLASSES is *configured* by checking that it's a tuple of strings
and is there. It's used/setup much later when those classes get imported
and initialised. It opens up more potential to mess around with this second
step - freeing MIDDLEWARE_CLASSES from being a tuple of dotted paths to
being, well, a set of classes. Circular imports are an issue, but IMO it
wouldn't be *that* awful to have the ProjectConfig file recommended that
you just put the imports in the customised methods all the time to be on
the safe side.
e.g.
MyProjectConfig(ProjectConfig):
def get_middleware_classes(self):
from thirdparty.middleware import ConfigurableMiddleware
classes = super().get_middleware_classes()
config = {...}
classes.append(ConfigurableMiddleware(**config)
return classes
Obviously you could not call the super at all (which inspects the setting),
do all the importing and initialisation of the default middleware yourself
and BOOM no settings.
[There are some technical issues to solve here to do with how you actually
access the built middleware classes from code, especially regarding
potential for changing them at runtime in tests]
Hypthetical: What we may find in this refactoring is that some of the
places Django currently dies if the settings are not configured can be
relaxed somewhat to not-at-import-time, so we may be able to avoid some of
this.
On 6 September 2014 22:04, Carl Meyer wrote:
> On 09/06/2014 03:10 AM, Jannis Leidel wrote:
> >> Yes, that approach would definitely require some thinking (which I
> >> haven't fully done yet) about how best to get a custom
> >> ProjectConfig inserted into both the wsgi.py and manage.py
> >> entry-points.
> >
> > Just a quick drive-by comment: ZOMG YES PLEASE.
> >
> > I also could see a default project class that uses a global settings
> > module that we’d provide for backwards compatibility, but that could
> > eventually be replaced by something that follows other configuration
> > styles.
> >
> > As to how to normalise the API between wsgi.py and manage.py I think
> > environment variables have worked well for not having to reach too
> > far into the WSGI and management code. So I suggest to introduce a
> > new env var called DJANGO_PROJECT with the dotted path to the Django
> > project object (or to something that quacks like one at least) that
> > defaults to ‘django.default_project’ or similar that does the
> > DJANGO_SETTINGS_MODULE inspection for django.conf.settings.
>
> I think we should avoid requiring a new environment variable, if
> possible, otherwise we'd just be repeating the unfortunate "you can't
> import this module because you haven't set DJANGO_SETTINGS_MODULE" type
> of error, and adding a new instance of "look up some class by dotted
> string", which is the sort of not-quite-Pythonic stuff that this
> proposal is aiming to avoid.
>
> Instead, I'd envision just providing an alternative new "run management
> command" API that did not call django.setup() for you, but expected you
> to have already called it. Then the stock manage.py could be updated to
> use that new API after calling django.setup() explicitly, and then it
> would be possible to pass in a custom ProjectConfig as an argument to
> django.setup() just by modifying your manage.py.
>
> At this point, wsgi.py and manage.py would not be special in any way;
> they would just be typical examples of the general rule that "if you
> want to write a script that uses Django, you should call django.setup()
> before doing anything else."
>
> All of this wouldn't really fix the circular-import issues we already
> have with importing stuff in settings, though. If you want to configure
> your ProjectConfig in code you'd need to import some classes (e.g.
> middleware), and you'd still need to be careful that that doesn't
> involve importing a module that has an import-time dependency on global
> config. So it would remain valuable to reduce import-time dependencies
> on the global config.
>
> In a sense, what a proposal like this would achieve is to merge the two
> current separate questions "are settings configured?" and "is the app
> cache ready?" into a single "is Django setup?". This is conceptually
> simpler, but I think it remains an open question how much practical
> improvement it achieves.
>
> Carl
>
>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAMwjO1H9RWufw69VtkOhR-ZoUuPrfo
Re: Setting dictionaries (was Re: integrating django-secure)
On 09/06/2014 03:10 AM, Jannis Leidel wrote: >> Yes, that approach would definitely require some thinking (which I >> haven't fully done yet) about how best to get a custom >> ProjectConfig inserted into both the wsgi.py and manage.py >> entry-points. > > Just a quick drive-by comment: ZOMG YES PLEASE. > > I also could see a default project class that uses a global settings > module that we’d provide for backwards compatibility, but that could > eventually be replaced by something that follows other configuration > styles. > > As to how to normalise the API between wsgi.py and manage.py I think > environment variables have worked well for not having to reach too > far into the WSGI and management code. So I suggest to introduce a > new env var called DJANGO_PROJECT with the dotted path to the Django > project object (or to something that quacks like one at least) that > defaults to ‘django.default_project’ or similar that does the > DJANGO_SETTINGS_MODULE inspection for django.conf.settings. I think we should avoid requiring a new environment variable, if possible, otherwise we'd just be repeating the unfortunate "you can't import this module because you haven't set DJANGO_SETTINGS_MODULE" type of error, and adding a new instance of "look up some class by dotted string", which is the sort of not-quite-Pythonic stuff that this proposal is aiming to avoid. Instead, I'd envision just providing an alternative new "run management command" API that did not call django.setup() for you, but expected you to have already called it. Then the stock manage.py could be updated to use that new API after calling django.setup() explicitly, and then it would be possible to pass in a custom ProjectConfig as an argument to django.setup() just by modifying your manage.py. At this point, wsgi.py and manage.py would not be special in any way; they would just be typical examples of the general rule that "if you want to write a script that uses Django, you should call django.setup() before doing anything else." All of this wouldn't really fix the circular-import issues we already have with importing stuff in settings, though. If you want to configure your ProjectConfig in code you'd need to import some classes (e.g. middleware), and you'd still need to be careful that that doesn't involve importing a module that has an import-time dependency on global config. So it would remain valuable to reduce import-time dependencies on the global config. In a sense, what a proposal like this would achieve is to merge the two current separate questions "are settings configured?" and "is the app cache ready?" into a single "is Django setup?". This is conceptually simpler, but I think it remains an open question how much practical improvement it achieves. Carl signature.asc Description: OpenPGP digital signature
Re: Setting dictionaries (was Re: integrating django-secure)
On 05 Sep 2014, at 23:18, Carl Meyer wrote:
> On 09/05/2014 03:11 PM, Marc Tamlyn wrote:
>> Yup, that works. The issue is that most users never explicitly call
>> django.setup(), so they need to know how to modify manage.py and wsgi.py
>> to do
>>
>> conf = MyProjectConfig()
>> django.setup(conf)
>>
>>
>> In the case of wsgi.py this isn't too obtuse, but django.setup() is here
>> -
>> https://github.com/django/django/blob/master/django/core/management/__init__.py#L310
>> for management commands. Not the easiest place to customise!
>
> Yes, that approach would definitely require some thinking (which I
> haven't fully done yet) about how best to get a custom ProjectConfig
> inserted into both the wsgi.py and manage.py entry-points.
Just a quick drive-by comment: ZOMG YES PLEASE.
I also could see a default project class that uses a global settings module
that we’d provide for backwards compatibility, but that could eventually be
replaced by something that follows other configuration styles.
As to how to normalise the API between wsgi.py and manage.py I think
environment variables have worked well for not having to reach too far into the
WSGI and management code. So I suggest to introduce a new env var called
DJANGO_PROJECT with the dotted path to the Django project object (or to
something that quacks like one at least) that defaults to
‘django.default_project’ or similar that does the DJANGO_SETTINGS_MODULE
inspection for django.conf.settings.
But if a different config style is needed, we’d override it:
DJANGO_PROJECT = ‘mysite.project.foo’
Default style:
import django
class FooProject(django.SettingsProject):
DEBUG = False
bar = FooProject()
Flask style config:
import django
import foo
bar = foo.ConfigProject()
bar.config.update(
DEBUG=True,
SECRET_KEY=‘...',
)
Or ini/yaml style config:
import django
import foo
bar = foo.YamlProject('/path/to/config.yaml')
Whether we’d want to ship the latter two project style classes as part of
Django is a different question, I just think that offering a common API for
project configuration (via a “Project” base classes) may be a sane way to
slowly move away from the Django settings file via duck typing.
Alternatively we could have the project itself be the wsgi app but I’m
reluctant to go down that road given the past refactoring around wsgi.py that
basically resulted in not making Django projects WSGI apps themselves. I like
the fact that we separate configuration from the WSGI gateway, but maybe
projects like Flask and Pyramid have shown that to be a non-issue after all.
I’m on the fence.
So far my quick brain dump.. :)
Jannis
signature.asc
Description: Message signed with OpenPGP using GPGMail
Re: Setting dictionaries (was Re: integrating django-secure)
On 09/05/2014 03:11 PM, Marc Tamlyn wrote: > Yup, that works. The issue is that most users never explicitly call > django.setup(), so they need to know how to modify manage.py and wsgi.py > to do > > conf = MyProjectConfig() > django.setup(conf) > > > In the case of wsgi.py this isn't too obtuse, but django.setup() is here > - > https://github.com/django/django/blob/master/django/core/management/__init__.py#L310 > for management commands. Not the easiest place to customise! Yes, that approach would definitely require some thinking (which I haven't fully done yet) about how best to get a custom ProjectConfig inserted into both the wsgi.py and manage.py entry-points. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/540A289A.9070509%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
Yup, that works. The issue is that most users never explicitly call django.setup(), so they need to know how to modify manage.py and wsgi.py to do conf = MyProjectConfig() django.setup(conf) In the case of wsgi.py this isn't too obtuse, but django.setup() is here - https://github.com/django/django/blob/master/django/core/management/__init__.py#L310 for management commands. Not the easiest place to customise! On 5 September 2014 22:00, Carl Meyer wrote: > On 09/05/2014 02:54 PM, Marc Tamlyn wrote: > > I like the idea Russ - from a brief peruse of the code base it's > > apparently to me how disparate the handling of various "setup" settings > > is throughout the codebase. For example: > > > > INSTALLED_APPS is handled by django.setup() and Apps.populate() (cached > > attribute) > > MIDDLEWARE_CLASSES is handled by BaseHandler.setup_middleware() (done > > once and cached) > > ROOT_URLCONF is handled by BaseHandler.get_response() (done every > request) > > DATABASES is handled by ConnectionHandler (cached property) > > DATABASE_ROUTERS is handled by ConnectionRouter (cached attribute) > > TEMPLATE_LOADERS is handled by find_template() (saved in a global) > > CACHES is handled by CacheHandler (saved in a threading.local()) > > > > etc. > > > > Perhaps there is an interesting possibility here to move some of these > > disparate "inspect settings, import or configure some stuff and store it > > for the duration of the process" onto a "global" object - something like > > ProjectConfig. It could have methods (cached properties?) like > > .get_middleware_classes() which would return the imported, configured > > middleware classes. > > I like all this. > > > We then (dun dun dn) introduce a setting > > PROJECT_CONFIG = 'django.project.ProjectConfig' which you can then set > > yourself if you want to do a more procedural approach. > > But I think this may be backwards; I prefer the idea of creating a > ProjectConfig yourself (via any method you like, you may not need to use > a settings module at all; you could totally get rid of > DJANGO_SETTINGS_MODULE if you want) and passing it as an optional > argument to django.setup(). > > Carl > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/540A2456.80601%40oddbird.net > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Hd%3DmKkvdUuUCN2Sr%2BHM-PY%3De5u8fqMGL5xY4jGNSwBBw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On 09/05/2014 02:54 PM, Marc Tamlyn wrote: > I like the idea Russ - from a brief peruse of the code base it's > apparently to me how disparate the handling of various "setup" settings > is throughout the codebase. For example: > > INSTALLED_APPS is handled by django.setup() and Apps.populate() (cached > attribute) > MIDDLEWARE_CLASSES is handled by BaseHandler.setup_middleware() (done > once and cached) > ROOT_URLCONF is handled by BaseHandler.get_response() (done every request) > DATABASES is handled by ConnectionHandler (cached property) > DATABASE_ROUTERS is handled by ConnectionRouter (cached attribute) > TEMPLATE_LOADERS is handled by find_template() (saved in a global) > CACHES is handled by CacheHandler (saved in a threading.local()) > > etc. > > Perhaps there is an interesting possibility here to move some of these > disparate "inspect settings, import or configure some stuff and store it > for the duration of the process" onto a "global" object - something like > ProjectConfig. It could have methods (cached properties?) like > .get_middleware_classes() which would return the imported, configured > middleware classes. I like all this. > We then (dun dun dn) introduce a setting > PROJECT_CONFIG = 'django.project.ProjectConfig' which you can then set > yourself if you want to do a more procedural approach. But I think this may be backwards; I prefer the idea of creating a ProjectConfig yourself (via any method you like, you may not need to use a settings module at all; you could totally get rid of DJANGO_SETTINGS_MODULE if you want) and passing it as an optional argument to django.setup(). Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/540A2456.80601%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
I like the idea Russ - from a brief peruse of the code base it's apparently to me how disparate the handling of various "setup" settings is throughout the codebase. For example: INSTALLED_APPS is handled by django.setup() and Apps.populate() (cached attribute) MIDDLEWARE_CLASSES is handled by BaseHandler.setup_middleware() (done once and cached) ROOT_URLCONF is handled by BaseHandler.get_response() (done every request) DATABASES is handled by ConnectionHandler (cached property) DATABASE_ROUTERS is handled by ConnectionRouter (cached attribute) TEMPLATE_LOADERS is handled by find_template() (saved in a global) CACHES is handled by CacheHandler (saved in a threading.local()) etc. Perhaps there is an interesting possibility here to move some of these disparate "inspect settings, import or configure some stuff and store it for the duration of the process" onto a "global" object - something like ProjectConfig. It could have methods (cached properties?) like .get_middleware_classes() which would return the imported, configured middleware classes. We then (dun dun dn) introduce a setting PROJECT_CONFIG = 'django.project.ProjectConfig' which you can then set yourself if you want to do a more procedural approach. In a sense, this does nothing. It doesn't remove global state (still lives in settings/project config), and it doesn't really change anything for most users. What it does do is start the "centralize" the configuration and use of settings, giving us 1) a way of changing how a setting is used, not just defined and 2) the option of not setting that setting at all and just doing it directly. This may (or may not!) interact quite nicely with override_settings() which simply does not work with some settings which are cached forever. What is also to me interesting about it is that we may be able to do quite a lot of the work without breaking any APIs at all - BaseHandler.setup_middleware() can stay, it just calls something else to do the work. On 5 September 2014 21:21, Carl Meyer wrote: > Hi Russ, > > On 09/05/2014 10:38 AM, Russell Keith-Magee wrote: > > The other way to resolve this would be to rethink the way Django starts > up. > > > > Settings like MIDDLEWARE_CLASSES have always struck me as being a little > > odd - they're not really settings, they're a way to configure the way > > code is executed without actually writing and executing code. The same > > is true of things like ROOT_URLCONF, a few of the template loaders and > > path specifications, and so on. > > > > If we were starting green-fields, it seems to me that we wouldn't do it > > this way; the Pyramid/Flask approach of doing this sort of thing by > > actually defining code would make a lot more sense. Django currently has > > a well defined, but opaque startup sequence where settings are loaded > > and then the middlware, urlconfs etc are set up. The alternative would > > be an explicit startup sequence that constructs the desired middlewares, > > installs them into a root urlconf, configures loaders and so on, and > > then sets the environment running. > > > > This should also avoids the circular dependency problem, because > > anything that needs to do an import in the settings file would be part > > of the configuration process. > > > > To be clear, I know this would be a huge change. If we went down this > > path, we'd need to maintain the old/current way of deploying Django for > > a long time. In the interests of getting started quickly, it might even > > be desirable to maintain both approaches long term. But having an > > explicit startup sequence would allow for complex middleware > > configuration like this thread has proposed. I'm just throwing this idea > > out on the porch to see if the cat licks it up. > > I think this is generally down a similar line as the proposals that have > been floated in the past (e.g. in a talk that Alex Gaynor gave at > DjangoCon(?) a few years ago) to move Django away from its reliance on > process-global configuration. > > Currently in Django a "project" is (implicitly) just "a settings > module". The way I could see your proposal happening would be to > introduce a Project (or Config?) class (in any other framework it would > be App, but in Django that name is taken) which exposes APIs for > imperative configuration of things like URLconf, middleware, etc, and > then is itself (or can provide) a WSGI application callable. > > Then any script which wants to "start" Django would have two options: > call django.setup(), which implements the current "look for a settings > module and configure an implicit Project based on those settings" > startup, or instantiate their own Project instead. > > The knottiest problem with moving away from process-global config is > that it allows for our current "simple" APIs (e.g. import a model class > and query on it, implicitly relying on global DATABASES config; import > "render_to_string" and render a template, implicitly relying on global > TEMPLATE_*
Re: Setting dictionaries (was Re: integrating django-secure)
Hi Ryan, On 09/05/2014 12:02 PM, Ryan Hiebert wrote: > I'd like to see the email settings get moved into a dictionary so that > I can use a single environment variable to configure it very easily, ala > dj-database-url. There's dj-email-url right now, but it requires 7 lines > to configure unless I want to restore to using vars() to update them all > at once. There should be a way to set this one logical setting (based on > the environment variable) without being implicit (using var().update) > or spending 7 lines on something this stupidly simple. This is the best reason I've seen mentioned for moving the email settings to a dictionary. It's true that email settings really are various aspects of configuring one "thing", like DATABASES or CACHES. I'm still not sure it's worth the backwards-compatibility break, though, especially since it is possible for a package like dj-email-url to do what you want with no boilerplate in your settings file, with some code in its AppConfig.ready() that looks for a custom dictionary-formatted setting and sets the actual individual settings based on it. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/540A1D7B.8000804%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
Hi Russ, On 09/05/2014 10:38 AM, Russell Keith-Magee wrote: > The other way to resolve this would be to rethink the way Django starts up. > > Settings like MIDDLEWARE_CLASSES have always struck me as being a little > odd - they're not really settings, they're a way to configure the way > code is executed without actually writing and executing code. The same > is true of things like ROOT_URLCONF, a few of the template loaders and > path specifications, and so on. > > If we were starting green-fields, it seems to me that we wouldn't do it > this way; the Pyramid/Flask approach of doing this sort of thing by > actually defining code would make a lot more sense. Django currently has > a well defined, but opaque startup sequence where settings are loaded > and then the middlware, urlconfs etc are set up. The alternative would > be an explicit startup sequence that constructs the desired middlewares, > installs them into a root urlconf, configures loaders and so on, and > then sets the environment running. > > This should also avoids the circular dependency problem, because > anything that needs to do an import in the settings file would be part > of the configuration process. > > To be clear, I know this would be a huge change. If we went down this > path, we'd need to maintain the old/current way of deploying Django for > a long time. In the interests of getting started quickly, it might even > be desirable to maintain both approaches long term. But having an > explicit startup sequence would allow for complex middleware > configuration like this thread has proposed. I'm just throwing this idea > out on the porch to see if the cat licks it up. I think this is generally down a similar line as the proposals that have been floated in the past (e.g. in a talk that Alex Gaynor gave at DjangoCon(?) a few years ago) to move Django away from its reliance on process-global configuration. Currently in Django a "project" is (implicitly) just "a settings module". The way I could see your proposal happening would be to introduce a Project (or Config?) class (in any other framework it would be App, but in Django that name is taken) which exposes APIs for imperative configuration of things like URLconf, middleware, etc, and then is itself (or can provide) a WSGI application callable. Then any script which wants to "start" Django would have two options: call django.setup(), which implements the current "look for a settings module and configure an implicit Project based on those settings" startup, or instantiate their own Project instead. The knottiest problem with moving away from process-global config is that it allows for our current "simple" APIs (e.g. import a model class and query on it, implicitly relying on global DATABASES config; import "render_to_string" and render a template, implicitly relying on global TEMPLATE_* config; import "send_mail" and send an email, implicitly relying on global EMAIL_* config). It would be possible to bite off the "imperative configuration" piece without the "kill global config" piece, though - we'd just need to provide an API (an alternative to django.setup(), or perhaps just an optional argument to it) that lets you install your own Project/Config instance as "the implicit global Project/Config", and then go ahead and use all the relying-on-global-config APIs just as you do now. Interesting to think about, but also a big chunk of very hypothetical work :-) Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/540A1B4B.8080009%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On Fri, Sep 5, 2014 at 12:49 PM, Russell Keith-Magee < russ...@keith-magee.com> wrote: > > On Fri, Sep 5, 2014 at 9:57 AM, Carl Meyer wrote: > >> On 09/05/2014 10:46 AM, Tim Graham wrote: >> > In the meantime, shall we abandon the idea of organizing settings in >> > dictionaries for "purity"? May we add new settings as described here: >> > https://code.djangoproject.com/ticket/22734#comment:22 >> >> IMO, yes. >> >> When we already have a subsystem, like email, which is configured via >> global settings, and it really needs a new configuration knob (we should >> continue to be generally conservative in making that assessment), I see >> very little downside to a new setting. If there's to be a new knob, best >> to put it in the same place as the existing knobs for that component. >> > > +1. If we actually need a new email setting for timeouts, I have no > objection to adding EMAIL_TIMEOUT. > I'd like to see the email settings get moved into a dictionary so that I can use a single environment variable to configure it very easily, ala dj-database-url. There's dj-email-url right now, but it requires 7 lines to configure unless I want to restore to using vars() to update them all at once. There should be a way to set this one logical setting (based on the environment variable) without being implicit (using var().update) or spending 7 lines on something this stupidly simple. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CABpHFHQ2Hn7JPqh7F9rzXz6CA1yr1hG3Bc1Zi%2BGXDkLzsBdOTQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On Fri, Sep 5, 2014 at 9:57 AM, Carl Meyer wrote: > On 09/05/2014 10:46 AM, Tim Graham wrote: > > In the meantime, shall we abandon the idea of organizing settings in > > dictionaries for "purity"? May we add new settings as described here: > > https://code.djangoproject.com/ticket/22734#comment:22 > > IMO, yes. > > When we already have a subsystem, like email, which is configured via > global settings, and it really needs a new configuration knob (we should > continue to be generally conservative in making that assessment), I see > very little downside to a new setting. If there's to be a new knob, best > to put it in the same place as the existing knobs for that component. > +1. If we actually need a new email setting for timeouts, I have no objection to adding EMAIL_TIMEOUT. Russ %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAJxq848rnfbaTD-DU0fZNOGLo79eJEh_oMoeagjXqDfmibncCg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On 09/05/2014 10:46 AM, Tim Graham wrote: > In the meantime, shall we abandon the idea of organizing settings in > dictionaries for "purity"? May we add new settings as described here: > https://code.djangoproject.com/ticket/22734#comment:22 IMO, yes. When we already have a subsystem, like email, which is configured via global settings, and it really needs a new configuration knob (we should continue to be generally conservative in making that assessment), I see very little downside to a new setting. If there's to be a new knob, best to put it in the same place as the existing knobs for that component. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5409EB8A.3080506%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
In the meantime, shall we abandon the idea of organizing settings in dictionaries for "purity"? May we add new settings as described here: https://code.djangoproject.com/ticket/22734#comment:22 On Friday, September 5, 2014 12:39:02 PM UTC-4, Russell Keith-Magee wrote: > > > > On Thu, Sep 4, 2014 at 4:39 PM, Carl Meyer > wrote: > >> On 09/04/2014 05:36 PM, Collin Anderson wrote: >> > I'm trying to think outside the box on settings. >> > >> > If we want to logically group settings together, what if we supported >> > syntax like this? >> > >> > MIDDLEWARE_CLASSES = ( >> > 'django.middleware.clickjacking.XFrameOptionsMiddleware', >> > SecurityMiddleware(content_type_nosniff=True, xss_filter=True), >> > ) >> > >> > It could be a more pythonic way for middleware to have their own >> settings. >> >> Proposals like this generally run afoul of circular import problems. As >> soon as you have people importing parts of Django or their project in >> their settings file, those modules likely import other modules which in >> turn import django.conf.settings, and boom! >> > > Since I'm sitting at the DjangoCon US sprints, and I'm feeling a little > optimistic and crazy, I thought I'd throw this idea out there... > > The other way to resolve this would be to rethink the way Django starts > up. > > Settings like MIDDLEWARE_CLASSES have always struck me as being a little > odd - they're not really settings, they're a way to configure the way code > is executed without actually writing and executing code. The same is true > of things like ROOT_URLCONF, a few of the template loaders and path > specifications, and so on. > > If we were starting green-fields, it seems to me that we wouldn't do it > this way; the Pyramid/Flask approach of doing this sort of thing by > actually defining code would make a lot more sense. Django currently has a > well defined, but opaque startup sequence where settings are loaded and > then the middlware, urlconfs etc are set up. The alternative would be an > explicit startup sequence that constructs the desired middlewares, installs > them into a root urlconf, configures loaders and so on, and then sets the > environment running. > > This should also avoids the circular dependency problem, because anything > that needs to do an import in the settings file would be part of the > configuration process. > > To be clear, I know this would be a huge change. If we went down this > path, we'd need to maintain the old/current way of deploying Django for a > long time. In the interests of getting started quickly, it might even be > desirable to maintain both approaches long term. But having an explicit > startup sequence would allow for complex middleware configuration like this > thread has proposed. I'm just throwing this idea out on the porch to see if > the cat licks it up. > > Yours, > Russ Magee %-) > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3df38529-9d9c-4828-9f5f-dec033d54d35%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On Thu, Sep 4, 2014 at 4:39 PM, Carl Meyer wrote: > On 09/04/2014 05:36 PM, Collin Anderson wrote: > > I'm trying to think outside the box on settings. > > > > If we want to logically group settings together, what if we supported > > syntax like this? > > > > MIDDLEWARE_CLASSES = ( > > 'django.middleware.clickjacking.XFrameOptionsMiddleware', > > SecurityMiddleware(content_type_nosniff=True, xss_filter=True), > > ) > > > > It could be a more pythonic way for middleware to have their own > settings. > > Proposals like this generally run afoul of circular import problems. As > soon as you have people importing parts of Django or their project in > their settings file, those modules likely import other modules which in > turn import django.conf.settings, and boom! > Since I'm sitting at the DjangoCon US sprints, and I'm feeling a little optimistic and crazy, I thought I'd throw this idea out there... The other way to resolve this would be to rethink the way Django starts up. Settings like MIDDLEWARE_CLASSES have always struck me as being a little odd - they're not really settings, they're a way to configure the way code is executed without actually writing and executing code. The same is true of things like ROOT_URLCONF, a few of the template loaders and path specifications, and so on. If we were starting green-fields, it seems to me that we wouldn't do it this way; the Pyramid/Flask approach of doing this sort of thing by actually defining code would make a lot more sense. Django currently has a well defined, but opaque startup sequence where settings are loaded and then the middlware, urlconfs etc are set up. The alternative would be an explicit startup sequence that constructs the desired middlewares, installs them into a root urlconf, configures loaders and so on, and then sets the environment running. This should also avoids the circular dependency problem, because anything that needs to do an import in the settings file would be part of the configuration process. To be clear, I know this would be a huge change. If we went down this path, we'd need to maintain the old/current way of deploying Django for a long time. In the interests of getting started quickly, it might even be desirable to maintain both approaches long term. But having an explicit startup sequence would allow for complex middleware configuration like this thread has proposed. I'm just throwing this idea out on the porch to see if the cat licks it up. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAJxq84_P%3D3_Wi8DLwav7qtiw5C249TEwAeLrJJnx4VXEkanNrg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On Thu, Sep 4, 2014 at 7:36 PM, Collin Anderson
wrote:
> I'm trying to think outside the box on settings.
>
> If we want to logically group settings together, what if we supported
> syntax like this?
>
> MIDDLEWARE_CLASSES = (
> 'django.middleware.clickjacking.XFrameOptionsMiddleware',
> SecurityMiddleware(content_type_nosniff=True, xss_filter=True),
> )
>
> It could be a more pythonic way for middleware to have their own settings.
>
This is likely to run afoul of circular imports. It would need to do
something like the following to provide init kwargs.
MIDDLEWARE_CLASSES = (
'django.middleware.clickjacking.XFrameOptionsMiddleware',
('django.middleware.security.SecurityMiddleware',
{'content_type_nosniff': True, 'xss_filter': True}),
)
Regards,
Michael Manfre
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAGdCwBuXspVUEb8QPkAJdiBL6cp-hf7F7Yh2v9jQevimZQo2Xw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
yeah... I thought of circular imports... you would need to lazy-load most imports. not fun. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5513d0a2-bc08-417f-b4f1-dea8e041d968%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
On 09/04/2014 05:36 PM, Collin Anderson wrote: > I'm trying to think outside the box on settings. > > If we want to logically group settings together, what if we supported > syntax like this? > > MIDDLEWARE_CLASSES = ( > 'django.middleware.clickjacking.XFrameOptionsMiddleware', > SecurityMiddleware(content_type_nosniff=True, xss_filter=True), > ) > > It could be a more pythonic way for middleware to have their own settings. Proposals like this generally run afoul of circular import problems. As soon as you have people importing parts of Django or their project in their settings file, those modules likely import other modules which in turn import django.conf.settings, and boom! Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5408F84D.2090901%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
Hi Shai, On 09/01/2014 01:15 AM, Shai Berger wrote: > This thread has had very little to do with django-secure for some time... Thanks :-) > On Sunday 31 August 2014 18:07:04 Carl Meyer wrote: >> >> In the case of the email settings, I think introducing a deprecation >> that requires people to update their settings files, for zero gain in >> capability, is a much bigger negative than any of the ones mentioned >> above, and should in itself be enough to scuttle the proposal. As I said >> on the ticket, if a significant new capability were introduced that >> required a change to the settings structure (e.g. multiple email >> backends configured at once, which is why CACHES and DATABASES are now >> dictionaries), that might provide enough benefit to justify a deprecation. >> > > A case in point is a change that was introduced in 1.7 -- putting the TEST > settings of databases into an inner dict. When it was brought up, all > responses were positive. The only negatives I've seen since then had to do > with the deprecation -- with the initial implementation, it was not to make a > settings file with test-settings which would work without warnings on both > 1.6 > and 1.7. This has been since corrected (by allowing new and old to co-exist > when they are equivalent). > > There was no gain in functionality, just the logical grouping -- and that, > itself, is somewhat limited (because the settings at issue were already in a > dictionary). There were some simplifications in backend code (esp. the Oracle > backend, which uses more of these settings than other backends). > > On the other hand, test settings are less popular for defaults in multiple- > level user settings, so some considerations may be different. > > As I said, everybody who commented on it back then liked it. I still like it > in that context (though, as I mostly work on the Oracle backend, I'm > biased). > If we now decide that we globally don't like the concept, perhaps it is not > too late to revert it. Or perhaps the decision shouldn't be so global. I would not say that I "globally don't like the concept"; just that it's not an obvious win in all cases; there are tradeoffs. So yes, I would advocate for "the decision shouldn't be so global", in either direction. I think a dictionary is good design in the case of DATABASES and CACHES, and there will probably be future cases where it is also well-justified. My objection in the case of the email settings is primarily that the benefit is not worth the backwards-incompatibility, and in the case of django-secure it's that the settings aren't closely related enough to justify the downsides. I'm slightly in favor of the database TEST_* change, despite the backwards incompatibility, for these reasons that don't apply in the other cases (roughly in order of importance): a) the TEST_* settings were rapidly expanding to become a full copy of the normal per-database settings, and it's logical that they would be: really you're just defining another database to be used in test. So there's an additional conceptual simplicity (and possibly code simplification too, though I haven't looked at the commit) achieved by making it look more like "just another database" configuration. b) The database TEST_* settings are, I think, less commonly used than the email settings, so the backwards-compatibility impact is smaller. c) The TEST_* settings are less likely to be overridden in a multiple-user-settings scenario (as you mentioned), and have no default values (so there's no issue with partial override of defaults). d) The settings were already contained within the DATABASES dictionary, so if you were trying to override them you were already dealing with the added complexity of overriding bits of dicts. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/54061434.3040808%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
Re: Setting dictionaries (was Re: integrating django-secure)
Le lundi 1 septembre 2014 09:15:21 UTC+2, Shai Berger a écrit : > > A case in point is a change that was introduced in 1.7 -- putting the TEST > settings of databases into an inner dict. When it was brought up, all > responses were positive. (...) (...) > As I said, everybody who commented on it back then liked it. I still like > it > in that context (though, as I mostly work on the Oracle backend, I'm > biased). > If we now decide that we globally don't like the concept, perhaps it is > not > too late to revert it. Or perhaps the decision shouldn't be so global. > When the dictionary setting is not initially populated in global_settings.py, this is less of a problem. So a rule *might* be: no dictionary setting if Django has default values in it. Even if I'm still +0 on the solution I committed. Claude -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/85931891-53df-4ff8-99ad-5a1a585a1a19%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
