Re: Making Django more PaaS-friendly

[Martin Owens]

I look after many different django websites and I'd like to add something 
conceptual.

There is a difference between settings intended by developers to be used by 
other developers in order to enable and configure their app's behaviour so 
it work with other code. (I call these developer settings)
And settings used by systems administrators to define the environment they 
are deployed to as well as any deployment specific controls. (I call these 
sysadmin settings)

Django has always mixed the two into one file, and it causes problems. I've 
seen keys and passwords committed to github, extravagant json or os.env 
based solutions with piles of lines of messy code to what I see most which 
is a local_settings.py with a template version which is copied in at 
runtime with sane defaults.

I'd be happy to see a canonical decision made for django, but I'd focus on 
"who should be editing this setting" not just what it's for, if it's a 
secret or if it should be different or the same per instance.

Best regards, Martin Owens

On Monday, April 11, 2016 at 7:33:54 AM UTC+1, James Bennett wrote:
>
> Apologies for how late in the process this is; job hunt + moving 
> cross-country ate a lot of my time.
>
> At Django Under the Hood I proposed that for Django 1.10 we try to find 
> some common best practices for deploying Django on popular 
> platform-as-a-service (PaaS) solutions, and building support for them into 
> Django. The biggest one of these is probably having the ability to read 
> configuration from environment variables instead of hard-coding things into 
> settings files.
>
> At the very least I'd like to propose (assuming Kenneth is on board with 
> it) integrating dj-database-url[1] or something like it directly into 
> Django, so that there's no longer a third-party dependency for reading the 
> database configuration from an environment variable. Whether this is just 
> porting dj-database-url itself in, or making the DATABASES setting 
> understand URLs, I'm unsure of yet and would be interested to hear feedback 
> on. Either way I'm willing to put in the time to develop the patch.
>
> More generally, I'd like to see Django grow helpers for specifying 
> settings that should be read from environment variables, and which 
> environment variables to use (the email settings are another common case, 
> as is anything that requires an API key to access).
>
> There are a few ways to design this. One option would be just a minimal 
> wrapper around os.getenv, perhaps taking an optional type or type-coercing 
> function, so that it would be possible in a settings file to do:
>
> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
>
> However, this is not much better than the current practice of calling 
> os.getenv. A better solution might be the ability to specify a group of 
> settings which will be read from the environment, and have Django 
> automatically read and set them. For example:
>
> ENV_SETTINGS = [
> ('SECRET_NUMBER_SETTING', int),
> ('ACME_API_KEY', str),
> ('VOLCANO_LAIR_PASSWORD', str),
> ]
>
> would read the named settings from those environment variables, and coerce 
> them to the appropriate types using the function provided.
>
> The main problem with this is that it's really not very elegant. But at 
> the moment I can't think of anything better, and so I'd like to throw the 
> floor open to ideas on nicer approaches to this. If one can't be found, I 
> do think Django 1.10 should at least figure out how to handle database 
> config from env since that's such a common use case nowadays, but ideally 
> we'd be able to pin down a good API for generically pulling configuration 
> from the environment.
>
>
> [1] https://github.com/kennethreitz/dj-database-url
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f0ff81bc-0eb0-476e-8741-6ba7e7db7926%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Joey Wilhelm]

Just to throw another voice in here, I'm currently using django-environ[1],
as mentioned by Sean Brant. In addition, I'm using a settings setup based
on cookiecutter-django[2]. This means having my settings split into
`config.settings.common`, `config.settings.local`,
`config.settings.production`, for example. In common, I have things like
the following:

DATABASES = {
# Raises ImproperlyConfigured exception if DATABASE_URL not in
os.environ
'default': env.db("DATABASE_URL"),
}
EMAIL_BACKEND = env('DJANGO_EMAIL_BACKEND',
default='django.core.mail.backends.smtp.EmailBackend')
# ...etc...

These would obviously propagate up and be used in the production settings.
Then in the local settings, I have things like:

SECRET_KEY = env('DJANGO_SECRET_KEY', default='CHANGEME!!!')
EMAIL_BACKEND = env('DJANGO_EMAIL_BACKEND',
default='django.core.mail.backends.console.EmailBackend')
DATABASES = {
'default': env.db('DATABASE_URL',
default='postgis://django:supersecretpassword@localhost:5432/django'),
}
# ...etc...

I don't know if this is all necessarily the best way to do it, but it has
been working quite well for me. Everything is read out of a .env file, and
I provide a template.env file in my project to show which values need to be
set in there. `DJANGO_SETTINGS_MODULE` defaults to production, so that
missing environment keys will raise alarms by default, and then when
possible, those keys become optional in development settings.

-Joey

[1]: https://github.com/joke2k/django-environ
[2]: https://github.com/pydanny/cookiecutter-django

On Mon, Apr 11, 2016 at 10:39 AM, Curtis Maloney 
wrote:

> Just want to throw my 3c in here...
>
> Firstly, for reference, I wrote django-classy-settings to help with
> various settings related problems, one of which being env-based setting.
>
> https://github.com/funkybob/django-classy-settings
>
> As my env decorators are written to be class properties, they don't quite
> fit this discussion... however, things to consider:
>
> 1. All env vars can have a fallback value (the method in my case)
> 2. You can mark an env var as not having a fallback value, and it will
> raise an error at start up if not set.
> 3. Non-trivial value types are not easy.
>
> The URL series of settings helpers deal with many of these cases fairly
> well, as they allow us to specify a backend (protocol), username, password,
> host, port, and positional and keyword arguments.
>
> Registering more protocols is actually quite easy, all told.
>
> However, it becomes tricky when we allow multiples, such as database and
> cache backends.  Do we settle on, for instance, a pattern of
> "DJANGO_DATABASE_{name}="? Or some other scheme?
>
> Email settings are certainly another big one, IME... as well as AWS keys...
>
> --
> C
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/570BE142.3000209%40tinbrain.net
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADBkHdK5yhxWoDB6yygnFC%2BfmPCn3n6x%3DJoxbym2HuTJ3oyDVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Curtis Maloney]

On 12/04/16 04:26, Aymeric Augustin wrote:

On 11 Apr 2016, at 19:39, Curtis Maloney  wrote:

1. All env vars can have a fallback value (the method in my case)
2. You can mark an env var as not having a fallback value, and it will raise an 
error at start up if not set.


There’s an additional complexity here.

Usually, it’s best for settings read from environment variables:

a. To use a default value in dev, even if that means a small degradation in
functionality. This allows developers to start working on the project without
adding dozens of exports to their $VIRTUAL_ENV/bin/postactivate, and to add
just the values they need when they work on specific parts like integrations
with third-party systems.

b. To crash if no value is provided in prod, in order to catch configuration
errors upfront.

One might think of switching the behavior depending on settings.DEBUG, but
that won't work because the switch is required to load settings properly.


This is all true it's generally not an issue when using 
django-classy-settings as it was engineered to make it easy to switch 
between settings environments such as dev/staging/testing/production.



I’ve seen lots of wrapper that don’t add enough value to be worth it. These
days I just write:

# settings/base.py

...

# settings/dev.py

FOO_TOKEN = os.environ.get('FOO_TOKEN', '')

# settings/prod.py

FOO_TOKEN = os.environ['FOO_TOKEN']

It isn’t as dry as it could be, but at least it’s simple.


This is one thing d-c-s helps you avoid... by letting you declare it all 
in one file, inherit complex/dependent settings through classes, and

switch between those with an env var [or whatever mechanism you like]

However, this is slightly tangential to the actual discussion

--
C

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/570BED47.7040602%40tinbrain.net.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Aymeric Augustin]

> On 11 Apr 2016, at 19:39, Curtis Maloney  wrote:
> 
> 1. All env vars can have a fallback value (the method in my case)
> 2. You can mark an env var as not having a fallback value, and it will raise 
> an error at start up if not set.

There’s an additional complexity here.

Usually, it’s best for settings read from environment variables:

a. To use a default value in dev, even if that means a small degradation in
functionality. This allows developers to start working on the project without
adding dozens of exports to their $VIRTUAL_ENV/bin/postactivate, and to add
just the values they need when they work on specific parts like integrations
with third-party systems.

b. To crash if no value is provided in prod, in order to catch configuration
errors upfront.

One might think of switching the behavior depending on settings.DEBUG, but
that won't work because the switch is required to load settings properly.

I’ve seen lots of wrapper that don’t add enough value to be worth it. These
days I just write:

# settings/base.py

...

# settings/dev.py

FOO_TOKEN = os.environ.get('FOO_TOKEN', '')

# settings/prod.py

FOO_TOKEN = os.environ['FOO_TOKEN']

It isn’t as dry as it could be, but at least it’s simple.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A0573632-AE0B-4243-AA63-4462C5596D75%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Curtis Maloney]

Just want to throw my 3c in here...

Firstly, for reference, I wrote django-classy-settings to help with 
various settings related problems, one of which being env-based setting.


https://github.com/funkybob/django-classy-settings

As my env decorators are written to be class properties, they don't 
quite fit this discussion... however, things to consider:


1. All env vars can have a fallback value (the method in my case)
2. You can mark an env var as not having a fallback value, and it will 
raise an error at start up if not set.

3. Non-trivial value types are not easy.

The URL series of settings helpers deal with many of these cases fairly 
well, as they allow us to specify a backend (protocol), username, 
password, host, port, and positional and keyword arguments.


Registering more protocols is actually quite easy, all told.

However, it becomes tricky when we allow multiples, such as database and 
cache backends.  Do we settle on, for instance, a pattern of 
"DJANGO_DATABASE_{name}="? Or some other scheme?


Email settings are certainly another big one, IME... as well as AWS keys...

--
C

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/570BE142.3000209%40tinbrain.net.
For more options, visit https://groups.google.com/d/optout.

Re: is_authenticated as property

[Florian Apolloner]

On Monday, April 11, 2016 at 6:57:46 PM UTC+2, Tim Graham wrote:
>
> I cannot think of much we could do besides monkey-patching the custom-user 
> model.
>

I would not call checking and rewriting the class in __new__ 
monkey-patching, but…

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5addf885-05f7-4bdb-89c4-f07426e8b80f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: is_authenticated as property

[Tim Graham]

Florian has raised the issue of custom user models which define these 
methods, "Allowing custom user models to have a method is a security risk 
since all checks will now return true." I proposed a compatibility system 
check error to detect that situation to alert the user. Do you think the 
backwards incompatibility is justified here or do you prefer some other 
solution? I cannot think of much we could do besides monkey-patching the 
custom-user model.

On Thursday, December 3, 2015 at 3:54:39 AM UTC-5, Aymeric Augustin wrote:
>
> 2015-12-03 7:02 GMT+01:00 Josh Smeaton 
> :
>
>> I think we should be asking.. why not? If there's not a good reason not 
>> to, let's make it a callable and a property.
>>
>
> The original discussion dates back to 2008; back then I believe we were 
> slightly more resistant to change, especially when it came to complicating 
> things to increase user-friendliness ;-)
>
> -- 
> Aymeric.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/92b34bce-c875-45d1-b6b7-95b5b9857ea6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Sean Brant]

https://github.com/joke2k/django-environ has helpers for loading most of
the backends from urls. We use it with a docker based deployment at the
moment.

On Mon, Apr 11, 2016 at 8:46 AM, Ryan Hiebert  wrote:

> I'd love to see better support for PaaS configuration, especially
> 12-factor. We use Heroku, and I've been directly exposed to the challenges
> and had to come up with some of my own solutions. Here's some thoughts I
> have on the matter, in no particular order.
>
>
> The SECRET_KEY needs to _not_ be required for the `collectstatic` command.
> I haven't tracked down why it currently _is_ required, but it means that I
> have added this line to my project, solely so that my project can be built
> on Heroku.
>
> SECRET_KEY = os.environ.get('SECRET_KEY', 'not-so-secret')
>
> SECRET_KEY is effectively optional (though I'm careful to make sure it's
> set), and I don't see a good reason that should be. Note that environment
> variables aren't normally available during slug build, since changing
> environment variables doesn't re-build the project.
>
>
> I believe that the biggest current impediment to finding a great solution
> to loading Django settings is that the settings are based on a Module.
> Django-Configurations has done an excellent job of making that into a
> class-based API instead, which allows for mixins and inheritance. Having
> built-in support for this would be an excellent way to allow 3rd party
> solutions to build up settings programatically. In the case we are
> discussing, from the environment.
>
> I believe this would make it far easier for projects like Kristian's
> django-12factor and dj-email-url to work well. dj-email-url is a good idea
> (put all the settings in 1 variable, like dj-database-url), except that all
> the settings are split up at the top-level of the module, making what's
> conceptually one multi-facted setting a multi-line pain.
>
> One possibility would be extending `DJANGO_SETTINGS_MODULE` with an
> optional class, separated with a colon, so to get a django-configurations
> style class you'd use
> `DJANGO_SETTINGS_MODULE=myproject.settings:MySettings`. Or we could change
> the variable name `DJANGO_SETTINGS`, which would allow us to avoid the
> colon: `DJANGO_SETTINGS=myproject.settings.MySettings`. Or paint the
> bikeshed some other color.
>
>
> I'd love to see some of the support for environment configurations brought
> into Django, but IMO moving away from a module based settings API is a more
> pressing concern that would enable 3rd party libraries to come up with
> elegant solutions.
>
>
> On Apr 11, 2016, at 4:32 AM, Kristian Glass <
> googleto...@doismellburning.co.uk> wrote:
>
> I wrote django12factor to do something similar. One of the things I like
> least about it is the process of actually using it from your settings.py -
> there's things in there I'd love to see in the generated default.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/2D66F14E-DD71-41B6-A84D-A0C679FD72F5%40ryanhiebert.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAPNuhQxbRKX%3D2FrCd-jg6MJCnO%2BuBC6HbTOpPUdQJAjQjyMDSw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Ryan Hiebert]

I'd love to see better support for PaaS configuration, especially 12-factor. We 
use Heroku, and I've been directly exposed to the challenges and had to come up 
with some of my own solutions. Here's some thoughts I have on the matter, in no 
particular order.


The SECRET_KEY needs to _not_ be required for the `collectstatic` command. I 
haven't tracked down why it currently _is_ required, but it means that I have 
added this line to my project, solely so that my project can be built on Heroku.

SECRET_KEY = os.environ.get('SECRET_KEY', 'not-so-secret')

SECRET_KEY is effectively optional (though I'm careful to make sure it's set), 
and I don't see a good reason that should be. Note that environment variables 
aren't normally available during slug build, since changing environment 
variables doesn't re-build the project.


I believe that the biggest current impediment to finding a great solution to 
loading Django settings is that the settings are based on a Module. 
Django-Configurations has done an excellent job of making that into a 
class-based API instead, which allows for mixins and inheritance. Having 
built-in support for this would be an excellent way to allow 3rd party 
solutions to build up settings programatically. In the case we are discussing, 
from the environment.

I believe this would make it far easier for projects like Kristian's 
django-12factor and dj-email-url to work well. dj-email-url is a good idea (put 
all the settings in 1 variable, like dj-database-url), except that all the 
settings are split up at the top-level of the module, making what's 
conceptually one multi-facted setting a multi-line pain.

One possibility would be extending `DJANGO_SETTINGS_MODULE` with an optional 
class, separated with a colon, so to get a django-configurations style class 
you'd use `DJANGO_SETTINGS_MODULE=myproject.settings:MySettings`. Or we could 
change the variable name `DJANGO_SETTINGS`, which would allow us to avoid the 
colon: `DJANGO_SETTINGS=myproject.settings.MySettings`. Or paint the bikeshed 
some other color.


I'd love to see some of the support for environment configurations brought into 
Django, but IMO moving away from a module based settings API is a more pressing 
concern that would enable 3rd party libraries to come up with elegant solutions.


> On Apr 11, 2016, at 4:32 AM, Kristian Glass 
>  wrote:
> 
> I wrote django12factor to do something similar. One of the things I like 
> least about it is the process of actually using it from your settings.py - 
> there's things in there I'd love to see in the generated default.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2D66F14E-DD71-41B6-A84D-A0C679FD72F5%40ryanhiebert.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Kristian Glass]

Very very glad to hear this. All too frequently in #django, "please show us 
your settings (and remove any sensitive data)" ends up with a "Now you need 
to reset your SECRET_KEY" etc.

I wrote django12factor to do something similar. One of the things I like 
least about it is the process of actually using it from your settings.py - 
there's things in there I'd love to see in the generated default.

Right now it handles most settings individually and only addresses "the 
most common ones" - a simple boolean handler for DEBUG, string splitting 
for ALLOWED_HOSTS etc., and outsources CACHES / DATABASES / EMAIL* to 
django-cache-url, dj-database-url and dj-email-url respectively. This isn't 
ideal, BUT even without getting into generically useful helpers, I think a 
lot of those could be useful in the standard template...

https://github.com/doismellburning/django12factor/pull/19/files is a branch 
where I prototyped something similar to your type coercion idea; you may 
find it more elegant, ymmv - I'd be interested in your thoughts.

On Monday, April 11, 2016 at 7:33:54 AM UTC+1, James Bennett wrote:
>
> Apologies for how late in the process this is; job hunt + moving 
> cross-country ate a lot of my time.
>
> At Django Under the Hood I proposed that for Django 1.10 we try to find 
> some common best practices for deploying Django on popular 
> platform-as-a-service (PaaS) solutions, and building support for them into 
> Django. The biggest one of these is probably having the ability to read 
> configuration from environment variables instead of hard-coding things into 
> settings files.
>
> At the very least I'd like to propose (assuming Kenneth is on board with 
> it) integrating dj-database-url[1] or something like it directly into 
> Django, so that there's no longer a third-party dependency for reading the 
> database configuration from an environment variable. Whether this is just 
> porting dj-database-url itself in, or making the DATABASES setting 
> understand URLs, I'm unsure of yet and would be interested to hear feedback 
> on. Either way I'm willing to put in the time to develop the patch.
>
> More generally, I'd like to see Django grow helpers for specifying 
> settings that should be read from environment variables, and which 
> environment variables to use (the email settings are another common case, 
> as is anything that requires an API key to access).
>
> There are a few ways to design this. One option would be just a minimal 
> wrapper around os.getenv, perhaps taking an optional type or type-coercing 
> function, so that it would be possible in a settings file to do:
>
> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
>
> However, this is not much better than the current practice of calling 
> os.getenv. A better solution might be the ability to specify a group of 
> settings which will be read from the environment, and have Django 
> automatically read and set them. For example:
>
> ENV_SETTINGS = [
> ('SECRET_NUMBER_SETTING', int),
> ('ACME_API_KEY', str),
> ('VOLCANO_LAIR_PASSWORD', str),
> ]
>
> would read the named settings from those environment variables, and coerce 
> them to the appropriate types using the function provided.
>
> The main problem with this is that it's really not very elegant. But at 
> the moment I can't think of anything better, and so I'd like to throw the 
> floor open to ideas on nicer approaches to this. If one can't be found, I 
> do think Django 1.10 should at least figure out how to handle database 
> config from env since that's such a common use case nowadays, but ideally 
> we'd be able to pin down a good API for generically pulling configuration 
> from the environment.
>
>
> [1] https://github.com/kennethreitz/dj-database-url
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a324953c-95f0-4efa-8e22-993eb48096d1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django template 'if ... is' feature design

[Tobias Kunze]

On 08/04/16 01:40, Stephen Kelly wrote:
> Carl Meyer wrote:
> 
>> It might be worth adding a short documentation note. We largely want to
>> avoid documenting Python's behavior in the Django docs, but a short note
>> in the template `is` docs reminding people not to ever use `is` with
>> strings or numbers might be worthwhile.
> 
> Or to only use it with None, True or False.

I'd be in favor of a documentation note like that, or a link to Python
documentation of `is`. I agree about not duplicating Python
documentation, but a short note like that could help many people who
aren't that familiar with Python in general.

Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/57077644.20103%40cutebit.de.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Raphaël Barrois]

Hi James,

>From the experience on our projects, the ``CONFIG.getstr('db.password')`` 
>format works well in settings files:
- It makes it clear that this field is loaded from the environment, and what 
type is expected
- The setting is set and loaded in the same place
- It allows for type checking
- It handles loading other values

This is easier than automagically altering settings based on environment 
variables, as the user might set a value in
some part of its settings file, yet find it overridden by some later 
declaration of ``ENV_SETTINGS``.



In my opinion, a few important questions to address would be:
- What is the format of those environment variables? For instance, using a 
'DJANGO_' prefix will avoid name conflicts
- Can developers use this "environment configuration parser" tool to parse 
extra, custom settings; or are we limited to
  Django's needs?
- Should we only support "loading from the environment", or is it useful to 
allow loading from a ini-style file
  (typically to override some values on a local development environment)?


We wrote a small helper to load configuration from various sources (environment 
and configuration files); this could be
helpful for designing this new feature: 
http://getconf.readthedocs.org/en/latest/

Feel free to ping me if you're interested in more feedback from building and 
using that kind of setup :)


On Sun, 10 Apr 2016 23:33:46 -0700
James Bennett  wrote:

> Apologies for how late in the process this is; job hunt + moving
> cross-country ate a lot of my time.
> 
> At Django Under the Hood I proposed that for Django 1.10 we try to find
> some common best practices for deploying Django on popular
> platform-as-a-service (PaaS) solutions, and building support for them into
> Django. The biggest one of these is probably having the ability to read
> configuration from environment variables instead of hard-coding things into
> settings files.
> 
> At the very least I'd like to propose (assuming Kenneth is on board with
> it) integrating dj-database-url[1] or something like it directly into
> Django, so that there's no longer a third-party dependency for reading the
> database configuration from an environment variable. Whether this is just
> porting dj-database-url itself in, or making the DATABASES setting
> understand URLs, I'm unsure of yet and would be interested to hear feedback
> on. Either way I'm willing to put in the time to develop the patch.
> 
> More generally, I'd like to see Django grow helpers for specifying settings
> that should be read from environment variables, and which environment
> variables to use (the email settings are another common case, as is
> anything that requires an API key to access).
> 
> There are a few ways to design this. One option would be just a minimal
> wrapper around os.getenv, perhaps taking an optional type or type-coercing
> function, so that it would be possible in a settings file to do:
> 
> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
> 
> However, this is not much better than the current practice of calling
> os.getenv. A better solution might be the ability to specify a group of
> settings which will be read from the environment, and have Django
> automatically read and set them. For example:
> 
> ENV_SETTINGS = [
> ('SECRET_NUMBER_SETTING', int),
> ('ACME_API_KEY', str),
> ('VOLCANO_LAIR_PASSWORD', str),
> ]
> 
> would read the named settings from those environment variables, and coerce
> them to the appropriate types using the function provided.
> 
> The main problem with this is that it's really not very elegant. But at the
> moment I can't think of anything better, and so I'd like to throw the floor
> open to ideas on nicer approaches to this. If one can't be found, I do
> think Django 1.10 should at least figure out how to handle database config
> from env since that's such a common use case nowadays, but ideally we'd be
> able to pin down a good API for generically pulling configuration from the
> environment.
> 
> 
> [1] https://github.com/kennethreitz/dj-database-url
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160411143238.6565a96a%40ithor.polyconseil.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Tim Graham]

Here is some past work on "Minimize the risk of SECRET_KEY leaks" [0] that 
was never completed: https://github.com/django/django/pull/2714

[0] https://code.djangoproject.com/ticket/20081

On Monday, April 11, 2016 at 6:51:38 AM UTC-4, Josh Smeaton wrote:
>
> I kind of like the idea of making all settings configurable via the 
> environment by prefixing with DJANGO_SETTINGNAME. Sort of like how click 
> allows environment variables for options: 
> http://click.pocoo.org/5/options/#values-from-environment-variables. 
> Ideally configuring settings from the environment should be as transparent 
> as possible to the settings file, so that users aren't required to 
> os.getenv('NAME', 'default') all over the place to derive value.
>
> Django uses quite a few dicts and lists though, so consideration would 
> need to be made for those. Perhaps double underscores could represent keys 
> within a dict?
>
> DJANGO_CACHES__default__BACKEND='redis'
>
> For lists we could split on whitespace (again like Click): 
>
> DJANGO_INSTALLED_APPS="contrib.admin contrib.auth".
>
> This does not address the type of the value though. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a334c7ec-5b60-43c9-8fe5-6d476e33debb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Josh Smeaton]

I kind of like the idea of making all settings configurable via the 
environment by prefixing with DJANGO_SETTINGNAME. Sort of like how click 
allows environment variables for 
options: http://click.pocoo.org/5/options/#values-from-environment-variables. 
Ideally configuring settings from the environment should be as transparent 
as possible to the settings file, so that users aren't required to 
os.getenv('NAME', 'default') all over the place to derive value.

Django uses quite a few dicts and lists though, so consideration would need 
to be made for those. Perhaps double underscores could represent keys 
within a dict?

DJANGO_CACHES__default__BACKEND='redis'

For lists we could split on whitespace (again like Click): 

DJANGO_INSTALLED_APPS="contrib.admin contrib.auth".

This does not address the type of the value though. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1af1c252-e17a-4e05-816c-c31493e7bcac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Aymeric Augustin]

We have a chicken’n’egg problem to support third party database backends with 
dj-database-url’s current API:

- It is each backend’s responsibility to parse the URL and extract the 
configuration for a given database.
- Parsing the URL is a prerequisite to figure which database backend to pick 
for parsing the URL.

I think we need a specific convenience API such as:

DATABASES = {
‘default’: {
‘NAME’: ‘python.path.to.backend’,
‘URL’: ‘backend://...',
}
}

-- 
Aymeric.

> On 11 Apr 2016, at 11:27, Shai Berger  wrote:
> 
> Django-database-url is very nice, but AFAICT has no way to support 3rd-party 
> backends. I think it needs to grow this support before it can be used in 
> Django. We can adapt the backend API to help, but it's a little tricky if we 
> don't want to import unused backend.
> 
> On 11 באפריל 2016 11:13:23 GMT+03:00, Marc Tamlyn  > wrote:
> I like the idea of integrating dj-database-url. There's a similar package for 
> CACHES[0], which is a neat idea but slightly more tricky to justify as I 
> don't know if PaaS tend to send the cache details in that format necessarily.
> 
> I'm not sure about the rest of the mail, although I'd definitely be up for 
> modifying the SECRET_KEY line in particular to make it strongly suggest that 
> this should be different per environment.
> 
> M
> 
> 
> [0] https://github.com/ghickman/django-cache-url 
> 
> 
> On 11 April 2016 at 07:33, James Bennett  > wrote:
> Apologies for how late in the process this is; job hunt + moving 
> cross-country ate a lot of my time.
> 
> At Django Under the Hood I proposed that for Django 1.10 we try to find some 
> common best practices for deploying Django on popular platform-as-a-service 
> (PaaS) solutions, and building support for them into Django. The biggest one 
> of these is probably having the ability to read configuration from 
> environment variables instead of hard-coding things into settings files.
> 
> At the very least I'd like to propose (assuming Kenneth is on board with it) 
> integrating dj-database-url[1] or something like it directly into Django, so 
> that there's no longer a third-party dependency for reading the database 
> configuration from an environment variable. Whether this is just porting 
> dj-database-url itself in, or making the DATABASES setting understand URLs, 
> I'm unsure of yet and would be interested to hear feedback on. Either way I'm 
> willing to put in the time to develop the patch.
> 
> More generally, I'd like to see Django grow helpers for specifying settings 
> that should be read from environment variables, and which environment 
> variables to use (the email settings are another common case, as is anything 
> that requires an API key to access).
> 
> There are a few ways to design this. One option would be just a minimal 
> wrapper around os.getenv, perhaps taking an optional type or type-coercing 
> function, so that it would be possible in a settings file to do:
> 
> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
> 
> However, this is not much better than the current practice of calling 
> os.getenv. A better solution might be the ability to specify a group of 
> settings which will be read from the environment, and have Django 
> automatically read and set them. For example:
> 
> ENV_SETTINGS = [
> ('SECRET_NUMBER_SETTING', int),
> ('ACME_API_KEY', str),
> ('VOLCANO_LAIR_PASSWORD', str),
> ]
> 
> would read the named settings from those environment variables, and coerce 
> them to the appropriate types using the function provided.
> 
> The main problem with this is that it's really not very elegant. But at the 
> moment I can't think of anything better, and so I'd like to throw the floor 
> open to ideas on nicer approaches to this. If one can't be found, I do think 
> Django 1.10 should at least figure out how to handle database config from env 
> since that's such a common use case nowadays, but ideally we'd be able to pin 
> down a good API for generically pulling configuration from the environment.
> 
> 
> [1] https://github.com/kennethreitz/dj-database-url 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the 

Re: Making Django more PaaS-friendly

[Shai Berger]

Django-database-url is very nice, but AFAICT has no way to support 3rd-party 
backends. I think it needs to grow this support before it can be used in 
Django. We can adapt the backend API to help, but it's a little tricky if we 
don't want to import unused backend.

On 11 באפריל 2016 11:13:23 GMT+03:00, Marc Tamlyn  wrote:
>I like the idea of integrating dj-database-url. There's a similar
>package
>for CACHES[0], which is a neat idea but slightly more tricky to justify
>as
>I don't know if PaaS tend to send the cache details in that format
>necessarily.
>
>I'm not sure about the rest of the mail, although I'd definitely be up
>for
>modifying the SECRET_KEY line in particular to make it strongly suggest
>that this should be different per environment.
>
>M
>
>
>[0] https://github.com/ghickman/django-cache-url
>
>On 11 April 2016 at 07:33, James Bennett  wrote:
>
>> Apologies for how late in the process this is; job hunt + moving
>> cross-country ate a lot of my time.
>>
>> At Django Under the Hood I proposed that for Django 1.10 we try to
>find
>> some common best practices for deploying Django on popular
>> platform-as-a-service (PaaS) solutions, and building support for them
>into
>> Django. The biggest one of these is probably having the ability to
>read
>> configuration from environment variables instead of hard-coding
>things into
>> settings files.
>>
>> At the very least I'd like to propose (assuming Kenneth is on board
>with
>> it) integrating dj-database-url[1] or something like it directly into
>> Django, so that there's no longer a third-party dependency for
>reading the
>> database configuration from an environment variable. Whether this is
>just
>> porting dj-database-url itself in, or making the DATABASES setting
>> understand URLs, I'm unsure of yet and would be interested to hear
>feedback
>> on. Either way I'm willing to put in the time to develop the patch.
>>
>> More generally, I'd like to see Django grow helpers for specifying
>> settings that should be read from environment variables, and which
>> environment variables to use (the email settings are another common
>case,
>> as is anything that requires an API key to access).
>>
>> There are a few ways to design this. One option would be just a
>minimal
>> wrapper around os.getenv, perhaps taking an optional type or
>type-coercing
>> function, so that it would be possible in a settings file to do:
>>
>> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
>>
>> However, this is not much better than the current practice of calling
>> os.getenv. A better solution might be the ability to specify a group
>of
>> settings which will be read from the environment, and have Django
>> automatically read and set them. For example:
>>
>> ENV_SETTINGS = [
>> ('SECRET_NUMBER_SETTING', int),
>> ('ACME_API_KEY', str),
>> ('VOLCANO_LAIR_PASSWORD', str),
>> ]
>>
>> would read the named settings from those environment variables, and
>coerce
>> them to the appropriate types using the function provided.
>>
>> The main problem with this is that it's really not very elegant. But
>at
>> the moment I can't think of anything better, and so I'd like to throw
>the
>> floor open to ideas on nicer approaches to this. If one can't be
>found, I
>> do think Django 1.10 should at least figure out how to handle
>database
>> config from env since that's such a common use case nowadays, but
>ideally
>> we'd be able to pin down a good API for generically pulling
>configuration
>> from the environment.
>>
>>
>> [1] https://github.com/kennethreitz/dj-database-url
>>
>> --
>> You received this message because you are subscribed to the Google
>Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it,
>send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to
>django-developers@googlegroups.com.
>> Visit this group at
>https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>>
>https://groups.google.com/d/msgid/django-developers/CAL13Cg85fYj0i0ysxJGo2SDesqELMeA%2BtnKJ9cdpqNHmQ%3DX3Pg%40mail.gmail.com
>>
>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Django developers  (Contributions to Django itself)" group.
>To unsubscribe from this group and stop receiving emails from it, send
>an email to django-developers+unsubscr...@googlegroups.com.
>To post to this group, send email to
>django-developers@googlegroups.com.
>Visit this group at https://groups.google.com/group/django-developers.
>To view this discussion on the web visit

Re: Making Django more PaaS-friendly

[Marc Tamlyn]

I like the idea of integrating dj-database-url. There's a similar package
for CACHES[0], which is a neat idea but slightly more tricky to justify as
I don't know if PaaS tend to send the cache details in that format
necessarily.

I'm not sure about the rest of the mail, although I'd definitely be up for
modifying the SECRET_KEY line in particular to make it strongly suggest
that this should be different per environment.

M


[0] https://github.com/ghickman/django-cache-url

On 11 April 2016 at 07:33, James Bennett  wrote:

> Apologies for how late in the process this is; job hunt + moving
> cross-country ate a lot of my time.
>
> At Django Under the Hood I proposed that for Django 1.10 we try to find
> some common best practices for deploying Django on popular
> platform-as-a-service (PaaS) solutions, and building support for them into
> Django. The biggest one of these is probably having the ability to read
> configuration from environment variables instead of hard-coding things into
> settings files.
>
> At the very least I'd like to propose (assuming Kenneth is on board with
> it) integrating dj-database-url[1] or something like it directly into
> Django, so that there's no longer a third-party dependency for reading the
> database configuration from an environment variable. Whether this is just
> porting dj-database-url itself in, or making the DATABASES setting
> understand URLs, I'm unsure of yet and would be interested to hear feedback
> on. Either way I'm willing to put in the time to develop the patch.
>
> More generally, I'd like to see Django grow helpers for specifying
> settings that should be read from environment variables, and which
> environment variables to use (the email settings are another common case,
> as is anything that requires an API key to access).
>
> There are a few ways to design this. One option would be just a minimal
> wrapper around os.getenv, perhaps taking an optional type or type-coercing
> function, so that it would be possible in a settings file to do:
>
> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
>
> However, this is not much better than the current practice of calling
> os.getenv. A better solution might be the ability to specify a group of
> settings which will be read from the environment, and have Django
> automatically read and set them. For example:
>
> ENV_SETTINGS = [
> ('SECRET_NUMBER_SETTING', int),
> ('ACME_API_KEY', str),
> ('VOLCANO_LAIR_PASSWORD', str),
> ]
>
> would read the named settings from those environment variables, and coerce
> them to the appropriate types using the function provided.
>
> The main problem with this is that it's really not very elegant. But at
> the moment I can't think of anything better, and so I'd like to throw the
> floor open to ideas on nicer approaches to this. If one can't be found, I
> do think Django 1.10 should at least figure out how to handle database
> config from env since that's such a common use case nowadays, but ideally
> we'd be able to pin down a good API for generically pulling configuration
> from the environment.
>
>
> [1] https://github.com/kennethreitz/dj-database-url
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAL13Cg85fYj0i0ysxJGo2SDesqELMeA%2BtnKJ9cdpqNHmQ%3DX3Pg%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMwjO1Hd6JFntN49a35E-ktnXMf9pTWjisdh7fwYHC_-bbPPBA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Making Django more PaaS-friendly

[James Bennett]

Apologies for how late in the process this is; job hunt + moving
cross-country ate a lot of my time.

At Django Under the Hood I proposed that for Django 1.10 we try to find
some common best practices for deploying Django on popular
platform-as-a-service (PaaS) solutions, and building support for them into
Django. The biggest one of these is probably having the ability to read
configuration from environment variables instead of hard-coding things into
settings files.

At the very least I'd like to propose (assuming Kenneth is on board with
it) integrating dj-database-url[1] or something like it directly into
Django, so that there's no longer a third-party dependency for reading the
database configuration from an environment variable. Whether this is just
porting dj-database-url itself in, or making the DATABASES setting
understand URLs, I'm unsure of yet and would be interested to hear feedback
on. Either way I'm willing to put in the time to develop the patch.

More generally, I'd like to see Django grow helpers for specifying settings
that should be read from environment variables, and which environment
variables to use (the email settings are another common case, as is
anything that requires an API key to access).

There are a few ways to design this. One option would be just a minimal
wrapper around os.getenv, perhaps taking an optional type or type-coercing
function, so that it would be possible in a settings file to do:

SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)

However, this is not much better than the current practice of calling
os.getenv. A better solution might be the ability to specify a group of
settings which will be read from the environment, and have Django
automatically read and set them. For example:

ENV_SETTINGS = [
('SECRET_NUMBER_SETTING', int),
('ACME_API_KEY', str),
('VOLCANO_LAIR_PASSWORD', str),
]

would read the named settings from those environment variables, and coerce
them to the appropriate types using the function provided.

The main problem with this is that it's really not very elegant. But at the
moment I can't think of anything better, and so I'd like to throw the floor
open to ideas on nicer approaches to this. If one can't be found, I do
think Django 1.10 should at least figure out how to handle database config
from env since that's such a common use case nowadays, but ideally we'd be
able to pin down a good API for generically pulling configuration from the
environment.


[1] https://github.com/kennethreitz/dj-database-url

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAL13Cg85fYj0i0ysxJGo2SDesqELMeA%2BtnKJ9cdpqNHmQ%3DX3Pg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.