Re: Making startproject's settings more 12-factor-y

[Javier Buzzi]

Think of it like this, you make a simple settings/base.py where you define 
something like this:

from django.conf import from_env

MEDIA_PATH = '/mnt/media'  # hardcoded to show you don't have to use `
from_env`

DEBUG = from_env.bool('DEBUG')

SECRET_KEY = from_env('SECRET_KEY')

DATABASES = {
'default': from_env.db('POSTGRES_URL'),  # 
"postgres://user:pass@db.server:4321/db_name?params"
'extra': from_env.db('SQLITE_URL', 
default='sqlite:tmp/my-tmp-sqlite.db')
}

CACHES = {
'default': from_env.cache('MEMCACHE_URL'),
'redis': from_env.cache('REDIS_URL')
}

..

and you can still have an other settings/dev.py or settings/test.py that 
all inherit from settings/base.py and still can customize it some more in 
those files. I have a project that because of ops, environment variables 
are an issue and i have to rely on having multiple environment files, i 
dont like it, but i live with it, such is life. I much rather have a 
base.py that ALL environments use, and then have a tests.py that inherits 
from base.py and will add mocks to things like s3 and other things needed 
to ensure tests pass. But i would really recommend to keep the amount of 
environment files to an absolute minimum -- nevertheless this doesnt rob 
you of being able to accomplish what you're used to doing.

- Buzzi

On Thursday, June 25, 2020 at 4:12:26 PM UTC-4, Bobby Mozumder wrote:
>
> If you at least don’t separate all variables that are dependent on the 
> environment from the settings.py, then you’re going to have to edit your 
> settings.py file anyways, defeating the purpose of this. Database and Cache 
> connection settings are clearly dependent on the environment.
>
> -bobby
>
> On Jun 25, 2020, at 1:51 PM, Kit La Touche > 
> wrote:
>
> Wow, `distutils.util.strtobool` is great to know about!
>
> So, can we refocus this conversation? This is starting to look like 
> previous conversations on this topic, which pull in a lot of possibilities 
> but don't lead to a change. How do we go about generating a DEP or other 
> consensus-building tool on what we want here?
>
> It seems to me this conversation has historically gotten stuck by trying 
> to bite off a bigger bite. Therefore, I would recommend a minimal change 
> that gestures towards the direction we want to explore.
>
> Personally, I think that *at minimum* providing Django-builtin "get from 
> env"  helpers would be great; beyond that, I'd love to have them be 
> included around `DEBUG` and `SECRET_KEY` with the current values as 
> defaults, so they're optional. Once we see how this gets used, we can see 
> about passing it a file instead of `os.environ`, or borrowing other ideas 
> from any of the various supporting projects that have been suggested.
>
> It's clear that different people have different use-cases and different 
> needs, but regardless, I think that it's clear also that including values 
> like DEBUG and SECRET_KEY as *hard coded values in settings* by default 
> does not point people towards good practices. What "good practices" are is 
> likely to differ in each person's case, but I think that suggesting one 
> option (again, my vote is "look in the environment") will at least help 
> newer devs understand that this is a topic they should learn more about.
>
> Thanks,
>
> --Kit
>
> On Thu, Jun 25, 2020 at 11:16 AM Javier Buzzi  > wrote:
>
>> Hi Bobby, yes, thank you, this looks around the line of what i would like 
>> us to implement in Django. 
>>
>> Side note: i saw this config('DEBUG', default=False, cast=bool) and 
>> thought "there is NO WAY that works", that led me to from distutils.util 
>> import strtobool, absolute mind blown! Thanks!
>>
>> -Buzzi
>>
>> On Thursday, June 25, 2020 at 1:03:19 PM UTC-4, Bobby Mozumder wrote:
>>>
>>> There’s also python-decouple that I use that I haven’t seen mentioned in 
>>> this thread. It lets you set specific environment variables in a separate 
>>> .env file or INI file: https://github.com/henriquebastos/python-decouple
>>>
>>> -bobby
>>>
>>> On Jun 25, 2020, at 4:47 AM, Javier Buzzi  wrote:
>>>
>>> Hey Tom, cool project haven't heard of it, looks to me more inline with 
>>> validating and converting user input from an api/form. I could really see 
>>> myself using this in my personal projects, however this looks like we'd be 
>>> going back to the class based configuration that im trying to avoid. 
>>> Nonetheless thank you for the share!
>>>
>>> - Buzzi
>>>
>>> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:

 Javier, I just wanted to point out another option for configuration: 
 pydantic  - it offers a very 
 slick and intuitive interface for settings management across environments, 
 seamless handing of environment variables by using type hints, and so on. 
 I 
 wouldn't recommend it for anything other than large sites with complex 
 configurations, but it does work well for those, once you grapple with 

Re: Making startproject's settings more 12-factor-y

[Bobby Mozumder]

If you at least don’t separate all variables that are dependent on the 
environment from the settings.py, then you’re going to have to edit your 
settings.py file anyways, defeating the purpose of this. Database and Cache 
connection settings are clearly dependent on the environment.

-bobby

> On Jun 25, 2020, at 1:51 PM, Kit La Touche  wrote:
> 
> Wow, `distutils.util.strtobool` is great to know about!
> 
> So, can we refocus this conversation? This is starting to look like previous 
> conversations on this topic, which pull in a lot of possibilities but don't 
> lead to a change. How do we go about generating a DEP or other 
> consensus-building tool on what we want here?
> 
> It seems to me this conversation has historically gotten stuck by trying to 
> bite off a bigger bite. Therefore, I would recommend a minimal change that 
> gestures towards the direction we want to explore.
> 
> Personally, I think that at minimum providing Django-builtin "get from env"  
> helpers would be great; beyond that, I'd love to have them be included around 
> `DEBUG` and `SECRET_KEY` with the current values as defaults, so they're 
> optional. Once we see how this gets used, we can see about passing it a file 
> instead of `os.environ`, or borrowing other ideas from any of the various 
> supporting projects that have been suggested.
> 
> It's clear that different people have different use-cases and different 
> needs, but regardless, I think that it's clear also that including values 
> like DEBUG and SECRET_KEY as hard coded values in settings by default does 
> not point people towards good practices. What "good practices" are is likely 
> to differ in each person's case, but I think that suggesting one option 
> (again, my vote is "look in the environment") will at least help newer devs 
> understand that this is a topic they should learn more about.
> 
> Thanks,
> 
> --Kit
> 
> On Thu, Jun 25, 2020 at 11:16 AM Javier Buzzi  > wrote:
> Hi Bobby, yes, thank you, this looks around the line of what i would like us 
> to implement in Django. 
> 
> Side note: i saw this config('DEBUG', default=False, cast=bool) and thought 
> "there is NO WAY that works", that led me to from distutils.util import 
> strtobool, absolute mind blown! Thanks!
> 
> -Buzzi
> 
> On Thursday, June 25, 2020 at 1:03:19 PM UTC-4, Bobby Mozumder wrote:
> There’s also python-decouple that I use that I haven’t seen mentioned in this 
> thread. It lets you set specific environment variables in a separate .env 
> file or INI file: https://github.com/henriquebastos/python-decouple 
> 
> 
> -bobby
> 
>> On Jun 25, 2020, at 4:47 AM, Javier Buzzi > wrote:
>> 
>> Hey Tom, cool project haven't heard of it, looks to me more inline with 
>> validating and converting user input from an api/form. I could really see 
>> myself using this in my personal projects, however this looks like we'd be 
>> going back to the class based configuration that im trying to avoid. 
>> Nonetheless thank you for the share!
>> 
>> - Buzzi
>> 
>> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
>> Javier, I just wanted to point out another option for configuration: 
>> pydantic  - it offers a very slick and 
>> intuitive interface for settings management across environments, seamless 
>> handing of environment variables by using type hints, and so on. I wouldn't 
>> recommend it for anything other than large sites with complex 
>> configurations, but it does work well for those, once you grapple with how 
>> to integrate it with django's settings so they're all exposed as 
>> `settings.FOO`, and so on.
>> 
>> I don't think I would want to integrate anything like this into Django 
>> proper, but it might deserve a mention in the documentation.
>> 
>> Tom
>> 
>> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi > wrote:
>> This makes sense, I have a project that has a lot of settings files that get 
>> activated depending on the value of DJANGO_SETTINGS_MODULE. The solution i 
>> outlined above takes your reservations under consideration, if you want to 
>> use it, great, if not also great -- its a supplement not a requirement.
>> 
>> - Buzzi
>> 
>> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>>  tMost of the world is not as seamless as heroku.  My DevOps won't give me 
>> any more than a handful of environment variables.  I wanted something like 
>> DATABASE_URL, but all I have is DJANGO_LOG_DIR and DJANGO_SETTINGS_MODULE, 
>> and so I need many, many settings files. I think that happens a lot, and 
>> maybe a common pattern.
>> 
>> From a 12factor perspective, I would like to get it down to local settings 
>> (development) and production settings - yet for a lot of users, DevOps is 
>> not really supporting a full PaaS-like experience any way.
>> 
>> So - all of this has to be optional, which seems to rule out making it part 
>> of 

Re: Making startproject's settings more 12-factor-y

[Tom Forbes]

A small incremental change with some Django helpers and showing their use in 
the default settings.py is a great step forward, I would be +1 on that.

Tom

> On 25 Jun 2020, at 18:52, Kit La Touche  wrote:
> 
> 
> Wow, `distutils.util.strtobool` is great to know about!
> 
> So, can we refocus this conversation? This is starting to look like previous 
> conversations on this topic, which pull in a lot of possibilities but don't 
> lead to a change. How do we go about generating a DEP or other 
> consensus-building tool on what we want here?
> 
> It seems to me this conversation has historically gotten stuck by trying to 
> bite off a bigger bite. Therefore, I would recommend a minimal change that 
> gestures towards the direction we want to explore.
> 
> Personally, I think that at minimum providing Django-builtin "get from env"  
> helpers would be great; beyond that, I'd love to have them be included around 
> `DEBUG` and `SECRET_KEY` with the current values as defaults, so they're 
> optional. Once we see how this gets used, we can see about passing it a file 
> instead of `os.environ`, or borrowing other ideas from any of the various 
> supporting projects that have been suggested.
> 
> It's clear that different people have different use-cases and different 
> needs, but regardless, I think that it's clear also that including values 
> like DEBUG and SECRET_KEY as hard coded values in settings by default does 
> not point people towards good practices. What "good practices" are is likely 
> to differ in each person's case, but I think that suggesting one option 
> (again, my vote is "look in the environment") will at least help newer devs 
> understand that this is a topic they should learn more about.
> 
> Thanks,
> 
> --Kit
> 
>> On Thu, Jun 25, 2020 at 11:16 AM Javier Buzzi  wrote:
>> Hi Bobby, yes, thank you, this looks around the line of what i would like us 
>> to implement in Django. 
>> 
>> Side note: i saw this config('DEBUG', default=False, cast=bool) and thought 
>> "there is NO WAY that works", that led me to from distutils.util import 
>> strtobool, absolute mind blown! Thanks!
>> 
>> -Buzzi
>> 
>>> On Thursday, June 25, 2020 at 1:03:19 PM UTC-4, Bobby Mozumder wrote:
>>> There’s also python-decouple that I use that I haven’t seen mentioned in 
>>> this thread. It lets you set specific environment variables in a separate 
>>> .env file or INI file: https://github.com/henriquebastos/python-decouple
>>> 
>>> -bobby
>>> 
 On Jun 25, 2020, at 4:47 AM, Javier Buzzi  wrote:
 
 Hey Tom, cool project haven't heard of it, looks to me more inline with 
 validating and converting user input from an api/form. I could really see 
 myself using this in my personal projects, however this looks like we'd be 
 going back to the class based configuration that im trying to avoid. 
 Nonetheless thank you for the share!
 
 - Buzzi
 
> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
> Javier, I just wanted to point out another option for configuration: 
> pydantic - it offers a very slick and intuitive interface for settings 
> management across environments, seamless handing of environment variables 
> by using type hints, and so on. I wouldn't recommend it for anything 
> other than large sites with complex configurations, but it does work well 
> for those, once you grapple with how to integrate it with django's 
> settings so they're all exposed as `settings.FOO`, and so on.
> 
> I don't think I would want to integrate anything like this into Django 
> proper, but it might deserve a mention in the documentation.
> 
> Tom
> 
>> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi  wrote:
>> This makes sense, I have a project that has a lot of settings files that 
>> get activated depending on the value of DJANGO_SETTINGS_MODULE. The 
>> solution i outlined above takes your reservations under consideration, 
>> if you want to use it, great, if not also great -- its a supplement not 
>> a requirement.
>> 
>> - Buzzi
>> 
>>> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>>>  tMost of the world is not as seamless as heroku.  My DevOps won't give 
>>> me any more than a handful of environment variables.  I wanted 
>>> something like DATABASE_URL, but all I have is DJANGO_LOG_DIR and 
>>> DJANGO_SETTINGS_MODULE, and so I need many, many settings files. I 
>>> think that happens a lot, and maybe a common pattern.
>>> 
>>> From a 12factor perspective, I would like to get it down to local 
>>> settings (development) and production settings - yet for a lot of 
>>> users, DevOps is not really supporting a full PaaS-like experience any 
>>> way.
>>> 
>>> So - all of this has to be optional, which seems to rule out making it 
>>> part of the starting project template.  For sure, I've got my personal 

Re: Making startproject's settings more 12-factor-y

[Kit La Touche]

Wow, `distutils.util.strtobool` is great to know about!

So, can we refocus this conversation? This is starting to look like
previous conversations on this topic, which pull in a lot of possibilities
but don't lead to a change. How do we go about generating a DEP or other
consensus-building tool on what we want here?

It seems to me this conversation has historically gotten stuck by trying to
bite off a bigger bite. Therefore, I would recommend a minimal change that
gestures towards the direction we want to explore.

Personally, I think that *at minimum* providing Django-builtin "get from
env"  helpers would be great; beyond that, I'd love to have them be
included around `DEBUG` and `SECRET_KEY` with the current values as
defaults, so they're optional. Once we see how this gets used, we can see
about passing it a file instead of `os.environ`, or borrowing other ideas
from any of the various supporting projects that have been suggested.

It's clear that different people have different use-cases and different
needs, but regardless, I think that it's clear also that including values
like DEBUG and SECRET_KEY as *hard coded values in settings* by default
does not point people towards good practices. What "good practices" are is
likely to differ in each person's case, but I think that suggesting one
option (again, my vote is "look in the environment") will at least help
newer devs understand that this is a topic they should learn more about.

Thanks,

--Kit

On Thu, Jun 25, 2020 at 11:16 AM Javier Buzzi 
wrote:

> Hi Bobby, yes, thank you, this looks around the line of what i would like
> us to implement in Django.
>
> Side note: i saw this config('DEBUG', default=False, cast=bool) and
> thought "there is NO WAY that works", that led me to from distutils.util
> import strtobool, absolute mind blown! Thanks!
>
> -Buzzi
>
> On Thursday, June 25, 2020 at 1:03:19 PM UTC-4, Bobby Mozumder wrote:
>>
>> There’s also python-decouple that I use that I haven’t seen mentioned in
>> this thread. It lets you set specific environment variables in a separate
>> .env file or INI file: https://github.com/henriquebastos/python-decouple
>>
>> -bobby
>>
>> On Jun 25, 2020, at 4:47 AM, Javier Buzzi  wrote:
>>
>> Hey Tom, cool project haven't heard of it, looks to me more inline with
>> validating and converting user input from an api/form. I could really see
>> myself using this in my personal projects, however this looks like we'd be
>> going back to the class based configuration that im trying to avoid.
>> Nonetheless thank you for the share!
>>
>> - Buzzi
>>
>> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
>>>
>>> Javier, I just wanted to point out another option for configuration:
>>> pydantic  - it offers a very
>>> slick and intuitive interface for settings management across environments,
>>> seamless handing of environment variables by using type hints, and so on. I
>>> wouldn't recommend it for anything other than large sites with complex
>>> configurations, but it does work well for those, once you grapple with how
>>> to integrate it with django's settings so they're all exposed as
>>> `settings.FOO`, and so on.
>>>
>>> I don't think I would want to integrate anything like this into Django
>>> proper, but it might deserve a mention in the documentation.
>>>
>>> Tom
>>>
>>> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi  wrote:
>>>
 This makes sense, I have a project that has a lot of settings files
 that get activated depending on the value of DJANGO_SETTINGS_MODULE. The
 solution i outlined above takes your reservations under consideration, if
 you want to use it, great, if not also great -- its a supplement not a
 requirement.

 - Buzzi

 On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>
>  tMost of the world is not as seamless as heroku.  My DevOps won't
> give me any more than a handful of environment variables.  I wanted
> something like DATABASE_URL, but all I have is DJANGO_LOG_DIR and
> DJANGO_SETTINGS_MODULE, and so I need many, many settings files. I think
> that happens a lot, and maybe a common pattern.
>
> From a 12factor perspective, I would like to get it down to local
> settings (development) and production settings - yet for a lot of users,
> DevOps is not really supporting a full PaaS-like experience any way.
>
> So - all of this has to be optional, which seems to rule out making it
> part of the starting project template.  For sure, I've got my personal
> template, and work has an on-premise template and a Cloud template as well
> - but the department of developers doesn't always use these.  I find
> databases containing the tables for other projects, long after the models
> and migrations are gone, indicating a start by copy mode.
>
> On Wed, Jun 24, 2020 at 1:35 PM Kit La Touche 
> wrote:
>
>> 

Re: Admin accessibility

[Tom Carrick]

I've managed to have a quick look, it looks really promising. Having a
total number of issues to compare against would be good to avoid
regressions or introducing new issues. I do have one concern. To be viable,
it needs to live in the django repo somewhere. And we'd need a full admin
site to test against, with every admin feature (stacked / tabular inlines,
filtering, search, date filters, list editable, autocomplete, ...) to be
fully representative and catch everything. This seems like a really big
amount of stuff to add into the repo, that will need to be maintained, and
possibly an easy thing to forget about. Also the maintenance is tricky, we
should ideally not update this site too often or the total number of issues
won't be as easily comparable. I wonder if there's another way to handle
it, but I can't think of one.

I also imagine this would be pretty intensive to run, it would probably be
best to not run it by default and have a buildbot command to run this
manually for PRs where there are changes to the admin.

On Thu, 25 Jun 2020 at 02:15, Thibaud Colas  wrote:

> Hi Tom,
>
> Great to hear – no rush if you’re busy with other things. Here’s a quick
> proof of concept, with 20 different pages/scenarios, tested with Axe, HTML
> Code Sniffer, and Lighthouse:
> https://github.com/thibaudcolas/django_admin_tests.
>
> I’m not a big fan of Lighthouse personally, and I already had the rest of
> the tools set up, hence why I went with this. I spent a bit of time making
> a report out of the test results, which you can see an example of at
> https://thibaudcolas.github.io/django_admin_tests/. This is generated in
> Travis, currently set up to run those tests & regenerate the report from
> Django’s `master` branch once per week.
>
> I added instructions on the README to run this locally if anyone wants to
> try it out, and a few words about what to make of the report / issues. I
> didn’t look into what the issues were though.
>
> Let me know what you think, and I should be able to spend more time on
> this next week if it helps.
>
> On Wednesday, 24 June 2020 at 09:11:52 UTC+1 t...@carrick.eu wrote:
>
>> A quick update first. I'm pretty busy with a combination of day job and
>> personal projects, but I should have time to start writing the draft DEP in
>> the next week or two.
>>
>> Thibaud, I think the absolute most important thing is a way to measure
>> progress, even if - as others have mentioned - that measure is imperfect. I
>> think getting a proof of concept of Lighthouse running against a few admin
>> pages would be extremely helpful. It's also probably one of the more
>> difficult things. If that seems like too much, I think catching what the CI
>> would miss is the next most important thing, so I think your #1 suggestion
>> would work well.
>>
>> On Tue, 23 Jun 2020 at 22:34, Thibaud Colas  wrote:
>>
>>> Hey Tom,
>>>
>>> I wanted to check if there is anything we/I could do to help in the
>>> meantime? Whether that’s by starting to map or audit the Django admin, or
>>> setting up a sample CI pipeline with accessibility tests, or something else
>>> altogether. It’s a bit daunting to get started with this but I think I
>>> could realistically do one of:
>>>
>>> 1. Create a draft map of the Django admin UI for later manual auditing
>>> purposes.
>>> 2. Audi a specific part of Django and creating a ticket with concrete
>>> things to fix.
>>> 3. Set up static analysis for the CSS or HTML to look for common
>>> accessibility gotchas.
>>> 4. Sett up automated in-browser accessibility checks on some parts of
>>> Django to show what that might look like in a CI pipeline.
>>>
>>> … and document the steps along the way, and report back here or as a
>>> ticket.
>>>
>>> Cheers,
>>>
>>> Thibaud
>>>
>>> On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote:

 Some further thoughts...

 Mariusz, I don't think I agree that WCAG is massive. It can look a
 little large but I think that's mostly because each guideline is split into
 smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50
 success criteria for AA, many of which can be checked automatically, and
 many of those that can't don't apply to us as we don't use audio / video.
 There's also a nice quick reference
  that can be filtered to
 remove AAA and anything else unwanted. Once you drill down it can again get
 quite wordy, but it covers a lot. I think it's worth remembering here is
 that the point is not to strictly adhere to a standard for the sake of it,
 just to improve the overall accessibility, and WCAG is a useful and
 measurable tool to get us some of the way there.

 Thibaud, I more or less agree with everything there. I'm not sure
 developers should have to do full accessibility checks for their changes in
 the admin. Using a screen reader for example can be very frustrating and
 confusing if 

Re: Making startproject's settings more 12-factor-y

[Javier Buzzi]

Hi Bobby, yes, thank you, this looks around the line of what i would like 
us to implement in Django. 

Side note: i saw this config('DEBUG', default=False, cast=bool) and thought 
"there is NO WAY that works", that led me to from distutils.util import 
strtobool, 
absolute mind blown! Thanks!

-Buzzi

On Thursday, June 25, 2020 at 1:03:19 PM UTC-4, Bobby Mozumder wrote:
>
> There’s also python-decouple that I use that I haven’t seen mentioned in 
> this thread. It lets you set specific environment variables in a separate 
> .env file or INI file: https://github.com/henriquebastos/python-decouple
>
> -bobby
>
> On Jun 25, 2020, at 4:47 AM, Javier Buzzi  > wrote:
>
> Hey Tom, cool project haven't heard of it, looks to me more inline with 
> validating and converting user input from an api/form. I could really see 
> myself using this in my personal projects, however this looks like we'd be 
> going back to the class based configuration that im trying to avoid. 
> Nonetheless thank you for the share!
>
> - Buzzi
>
> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
>>
>> Javier, I just wanted to point out another option for configuration: 
>> pydantic  - it offers a very slick 
>> and intuitive interface for settings management across environments, 
>> seamless handing of environment variables by using type hints, and so on. I 
>> wouldn't recommend it for anything other than large sites with complex 
>> configurations, but it does work well for those, once you grapple with how 
>> to integrate it with django's settings so they're all exposed as 
>> `settings.FOO`, and so on.
>>
>> I don't think I would want to integrate anything like this into Django 
>> proper, but it might deserve a mention in the documentation.
>>
>> Tom
>>
>> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi  wrote:
>>
>>> This makes sense, I have a project that has a lot of settings files that 
>>> get activated depending on the value of DJANGO_SETTINGS_MODULE. The 
>>> solution i outlined above takes your reservations under consideration, if 
>>> you want to use it, great, if not also great -- its a supplement not a 
>>> requirement.
>>>
>>> - Buzzi
>>>
>>> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:

  tMost of the world is not as seamless as heroku.  My DevOps won't give 
 me any more than a handful of environment variables.  I wanted something 
 like DATABASE_URL, but all I have is DJANGO_LOG_DIR and 
 DJANGO_SETTINGS_MODULE, and so I need many, many settings files. I think 
 that happens a lot, and maybe a common pattern.

 From a 12factor perspective, I would like to get it down to local 
 settings (development) and production settings - yet for a lot of users, 
 DevOps is not really supporting a full PaaS-like experience any way.

 So - all of this has to be optional, which seems to rule out making it 
 part of the starting project template.  For sure, I've got my personal 
 template, and work has an on-premise template and a Cloud template as well 
 - but the department of developers doesn't always use these.  I find 
 databases containing the tables for other projects, long after the models 
 and migrations are gone, indicating a start by copy mode.

 On Wed, Jun 24, 2020 at 1:35 PM Kit La Touche  wrote:

> Carlton—thanks very much for the feedback. Javier—likewise. In 
> particular, the imagined API you describe above is very appealing to me: 
> start with `from_env` and then if you learn more about this and want to, 
> add in some `EnvFileLoader`.
>
> I want to make clear my motivation and agenda here: I have recently 
> had some conversations with newer devs about their experiences with 
> deployment of apps they're working on, and with a friend at Heroku about 
> his informal research into the problems people have with the same. One 
> recurring friction point (and this is not just on Heroku at all, to be 
> clear) is that there are a number of things that people *don't know 
> they need to configure* for a working deployment.
>
> There are four settings that are recurring particular gotchas that 
> people miss: the secret key, debug, static files, and databases. Static 
> files seems big and out of scope, databases seems adequately handled by 
> dj-database-url for most cases, and if your case is more complex, you'll 
> learn it, but the other two (secret key and debug) seemed easy enough to 
> flag as "you probably need to configure these!" with this sort of change 
> to 
> settings. This would be a first step towards shortening the distance from 
> `startproject` to a working deployment.
>
> Newer devs in particular have, based on my conversations and this 
> friend's research, been unlikely to (a) know that there are different 
> `startproject` templates, and (b) feel 

Re: Making startproject's settings more 12-factor-y

[Bobby Mozumder]

There’s also python-decouple that I use that I haven’t seen mentioned in this 
thread. It lets you set specific environment variables in a separate .env file 
or INI file: https://github.com/henriquebastos/python-decouple

-bobby

> On Jun 25, 2020, at 4:47 AM, Javier Buzzi  wrote:
> 
> Hey Tom, cool project haven't heard of it, looks to me more inline with 
> validating and converting user input from an api/form. I could really see 
> myself using this in my personal projects, however this looks like we'd be 
> going back to the class based configuration that im trying to avoid. 
> Nonetheless thank you for the share!
> 
> - Buzzi
> 
> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
> Javier, I just wanted to point out another option for configuration: pydantic 
>  - it offers a very slick and intuitive 
> interface for settings management across environments, seamless handing of 
> environment variables by using type hints, and so on. I wouldn't recommend it 
> for anything other than large sites with complex configurations, but it does 
> work well for those, once you grapple with how to integrate it with django's 
> settings so they're all exposed as `settings.FOO`, and so on.
> 
> I don't think I would want to integrate anything like this into Django 
> proper, but it might deserve a mention in the documentation.
> 
> Tom
> 
> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi > wrote:
> This makes sense, I have a project that has a lot of settings files that get 
> activated depending on the value of DJANGO_SETTINGS_MODULE. The solution i 
> outlined above takes your reservations under consideration, if you want to 
> use it, great, if not also great -- its a supplement not a requirement.
> 
> - Buzzi
> 
> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>  tMost of the world is not as seamless as heroku.  My DevOps won't give me 
> any more than a handful of environment variables.  I wanted something like 
> DATABASE_URL, but all I have is DJANGO_LOG_DIR and DJANGO_SETTINGS_MODULE, 
> and so I need many, many settings files. I think that happens a lot, and 
> maybe a common pattern.
> 
> From a 12factor perspective, I would like to get it down to local settings 
> (development) and production settings - yet for a lot of users, DevOps is not 
> really supporting a full PaaS-like experience any way.
> 
> So - all of this has to be optional, which seems to rule out making it part 
> of the starting project template.  For sure, I've got my personal template, 
> and work has an on-premise template and a Cloud template as well - but the 
> department of developers doesn't always use these.  I find databases 
> containing the tables for other projects, long after the models and 
> migrations are gone, indicating a start by copy mode.
> 
> On Wed, Jun 24, 2020 at 1:35 PM Kit La Touche > wrote:
> Carlton—thanks very much for the feedback. Javier—likewise. In particular, 
> the imagined API you describe above is very appealing to me: start with 
> `from_env` and then if you learn more about this and want to, add in some 
> `EnvFileLoader`.
> 
> I want to make clear my motivation and agenda here: I have recently had some 
> conversations with newer devs about their experiences with deployment of apps 
> they're working on, and with a friend at Heroku about his informal research 
> into the problems people have with the same. One recurring friction point 
> (and this is not just on Heroku at all, to be clear) is that there are a 
> number of things that people don't know they need to configure for a working 
> deployment.
> 
> There are four settings that are recurring particular gotchas that people 
> miss: the secret key, debug, static files, and databases. Static files seems 
> big and out of scope, databases seems adequately handled by dj-database-url 
> for most cases, and if your case is more complex, you'll learn it, but the 
> other two (secret key and debug) seemed easy enough to flag as "you probably 
> need to configure these!" with this sort of change to settings. This would be 
> a first step towards shortening the distance from `startproject` to a working 
> deployment.
> 
> Newer devs in particular have, based on my conversations and this friend's 
> research, been unlikely to (a) know that there are different `startproject` 
> templates, and (b) feel equipped to choose one, if they do know.
> 
> My hope is to make the smallest possible change to just start us moving 
> towards more clearly flagging, especially for newer devs, "these are things 
> that will need additional configuration in order to move from 'works on my 
> machine' to 'deployed'."
> 
> Towards that end, I thought that adding a "you might want to get this from 
> the env" helper would be a clear indication to a new dev that this is a 
> matter to even consider. Adding other configuration-getting options like 
> different secret-store file backends seems like a 

Re: Proposal: Drop dependency on pytz in favor of zoneinfo

[Kevin Henry]

> The point about the arithmetic semantics is a good one. I'm curious to 
> know how often this is actually a problem.
>
That’s an interesting question. I think that doing localized datetime 
arithmetic in Django is idiomatic, if not necessarily common. Let’s say we 
have a calendar application that allows users to create events and set 
reminders. A typical way to handle an event creation request would be to 
activate() the user’s timezone and then read a localized datetime from the 
Form. Now, to create reminders, you add a timedelta of a day, a month, etc. 
and then store that result in the database. This datetime could be 
different under the new semantics.


2. Add a feature flag that allows switching directly to zoneinfo that will 
> eventually default to True, and replace pytz with a shim *around pytz* 
> that raises warnings whenever you use something pytz-specific. That will 
> give people time to opt in to using zoneinfo with, hopefully, zero changes 
> in the intermediate. (Unfortunately, though, it doesn't allow for any 
> incremental migration like pytz_deprecation_shim does).
>

That is approximately what I was thinking. For the sake of example, I’ll 
present an opt-in approach modeled after the New Middleware transition 
(though I haven’t given the opt-in mechanism much thought):

- In version X, Django deprecates usage of pytz and introduces the TIMEZONE 
setting. It has the same meaning as TIME_ZONE, but the use of it signals 
that the user is opting into zoneinfo. If it’s set, Django does not use or 
assume pytz.
- If it’s not set, Django uses pytz and gives the user deprecation 
warnings. Those could come from a system check; or when Django datetime 
utilities are used; or via a shim around timezone objects; or something 
else.
- In version Y, pytz and TIME_ZONE usage is removed.

As you say, the main difference from your proposal is that there’s no way 
to mix pytz and zoneinfo. Trying to allow that would slow down and 
complicate the transition, and I'm just not seeing much of a benefit to 
outweigh that. (Just one perspective, of course...)

Cheers,
Kevin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9072bb07-9455-456e-b783-2834a84eab4an%40googlegroups.com.

Re: Making startproject's settings more 12-factor-y

[Javier Buzzi]

Hey Tom, cool project haven't heard of it, looks to me more inline with 
validating and converting user input from an api/form. I could really see 
myself using this in my personal projects, however this looks like we'd be 
going back to the class based configuration that im trying to avoid. 
Nonetheless thank you for the share!

- Buzzi

On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
>
> Javier, I just wanted to point out another option for configuration: 
> pydantic  - it offers a very slick 
> and intuitive interface for settings management across environments, 
> seamless handing of environment variables by using type hints, and so on. I 
> wouldn't recommend it for anything other than large sites with complex 
> configurations, but it does work well for those, once you grapple with how 
> to integrate it with django's settings so they're all exposed as 
> `settings.FOO`, and so on.
>
> I don't think I would want to integrate anything like this into Django 
> proper, but it might deserve a mention in the documentation.
>
> Tom
>
> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi  > wrote:
>
>> This makes sense, I have a project that has a lot of settings files that 
>> get activated depending on the value of DJANGO_SETTINGS_MODULE. The 
>> solution i outlined above takes your reservations under consideration, if 
>> you want to use it, great, if not also great -- its a supplement not a 
>> requirement.
>>
>> - Buzzi
>>
>> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>>>
>>>  tMost of the world is not as seamless as heroku.  My DevOps won't give 
>>> me any more than a handful of environment variables.  I wanted something 
>>> like DATABASE_URL, but all I have is DJANGO_LOG_DIR and 
>>> DJANGO_SETTINGS_MODULE, and so I need many, many settings files. I think 
>>> that happens a lot, and maybe a common pattern.
>>>
>>> From a 12factor perspective, I would like to get it down to local 
>>> settings (development) and production settings - yet for a lot of users, 
>>> DevOps is not really supporting a full PaaS-like experience any way.
>>>
>>> So - all of this has to be optional, which seems to rule out making it 
>>> part of the starting project template.  For sure, I've got my personal 
>>> template, and work has an on-premise template and a Cloud template as well 
>>> - but the department of developers doesn't always use these.  I find 
>>> databases containing the tables for other projects, long after the models 
>>> and migrations are gone, indicating a start by copy mode.
>>>
>>> On Wed, Jun 24, 2020 at 1:35 PM Kit La Touche  wrote:
>>>
 Carlton—thanks very much for the feedback. Javier—likewise. In 
 particular, the imagined API you describe above is very appealing to me: 
 start with `from_env` and then if you learn more about this and want to, 
 add in some `EnvFileLoader`.

 I want to make clear my motivation and agenda here: I have recently had 
 some conversations with newer devs about their experiences with deployment 
 of apps they're working on, and with a friend at Heroku about his informal 
 research into the problems people have with the same. One recurring 
 friction point (and this is not just on Heroku at all, to be clear) is 
 that 
 there are a number of things that people *don't know they need to 
 configure* for a working deployment.

 There are four settings that are recurring particular gotchas that 
 people miss: the secret key, debug, static files, and databases. Static 
 files seems big and out of scope, databases seems adequately handled by 
 dj-database-url for most cases, and if your case is more complex, you'll 
 learn it, but the other two (secret key and debug) seemed easy enough to 
 flag as "you probably need to configure these!" with this sort of change 
 to 
 settings. This would be a first step towards shortening the distance from 
 `startproject` to a working deployment.

 Newer devs in particular have, based on my conversations and this 
 friend's research, been unlikely to (a) know that there are different 
 `startproject` templates, and (b) feel equipped to choose one, if they do 
 know.

 My hope is to make the smallest possible change to just start us moving 
 towards more clearly flagging, especially for newer devs, "these are 
 things 
 that will need additional configuration in order to move from 'works on my 
 machine' to 'deployed'."

 Towards that end, I thought that adding a "you might want to get this 
 from the env" helper would be a clear indication to a new dev that this is 
 a matter to even consider. Adding other configuration-getting options like 
 different secret-store file backends seems like a good next step.

 Thanks,

 --Kit

 On Wed, Jun 24, 2020 at 11:13 AM Javier Buzzi  
 wrote:

Re: Making startproject's settings more 12-factor-y

[Tom Carrick]

Javier, I just wanted to point out another option for configuration:
pydantic  - it offers a very slick
and intuitive interface for settings management across environments,
seamless handing of environment variables by using type hints, and so on. I
wouldn't recommend it for anything other than large sites with complex
configurations, but it does work well for those, once you grapple with how
to integrate it with django's settings so they're all exposed as
`settings.FOO`, and so on.

I don't think I would want to integrate anything like this into Django
proper, but it might deserve a mention in the documentation.

Tom

On Wed, 24 Jun 2020 at 23:52, Javier Buzzi  wrote:

> This makes sense, I have a project that has a lot of settings files that
> get activated depending on the value of DJANGO_SETTINGS_MODULE. The
> solution i outlined above takes your reservations under consideration, if
> you want to use it, great, if not also great -- its a supplement not a
> requirement.
>
> - Buzzi
>
> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>>
>>  tMost of the world is not as seamless as heroku.  My DevOps won't give
>> me any more than a handful of environment variables.  I wanted something
>> like DATABASE_URL, but all I have is DJANGO_LOG_DIR and
>> DJANGO_SETTINGS_MODULE, and so I need many, many settings files. I think
>> that happens a lot, and maybe a common pattern.
>>
>> From a 12factor perspective, I would like to get it down to local
>> settings (development) and production settings - yet for a lot of users,
>> DevOps is not really supporting a full PaaS-like experience any way.
>>
>> So - all of this has to be optional, which seems to rule out making it
>> part of the starting project template.  For sure, I've got my personal
>> template, and work has an on-premise template and a Cloud template as well
>> - but the department of developers doesn't always use these.  I find
>> databases containing the tables for other projects, long after the models
>> and migrations are gone, indicating a start by copy mode.
>>
>> On Wed, Jun 24, 2020 at 1:35 PM Kit La Touche  wrote:
>>
>>> Carlton—thanks very much for the feedback. Javier—likewise. In
>>> particular, the imagined API you describe above is very appealing to me:
>>> start with `from_env` and then if you learn more about this and want to,
>>> add in some `EnvFileLoader`.
>>>
>>> I want to make clear my motivation and agenda here: I have recently had
>>> some conversations with newer devs about their experiences with deployment
>>> of apps they're working on, and with a friend at Heroku about his informal
>>> research into the problems people have with the same. One recurring
>>> friction point (and this is not just on Heroku at all, to be clear) is that
>>> there are a number of things that people *don't know they need to
>>> configure* for a working deployment.
>>>
>>> There are four settings that are recurring particular gotchas that
>>> people miss: the secret key, debug, static files, and databases. Static
>>> files seems big and out of scope, databases seems adequately handled by
>>> dj-database-url for most cases, and if your case is more complex, you'll
>>> learn it, but the other two (secret key and debug) seemed easy enough to
>>> flag as "you probably need to configure these!" with this sort of change to
>>> settings. This would be a first step towards shortening the distance from
>>> `startproject` to a working deployment.
>>>
>>> Newer devs in particular have, based on my conversations and this
>>> friend's research, been unlikely to (a) know that there are different
>>> `startproject` templates, and (b) feel equipped to choose one, if they do
>>> know.
>>>
>>> My hope is to make the smallest possible change to just start us moving
>>> towards more clearly flagging, especially for newer devs, "these are things
>>> that will need additional configuration in order to move from 'works on my
>>> machine' to 'deployed'."
>>>
>>> Towards that end, I thought that adding a "you might want to get this
>>> from the env" helper would be a clear indication to a new dev that this is
>>> a matter to even consider. Adding other configuration-getting options like
>>> different secret-store file backends seems like a good next step.
>>>
>>> Thanks,
>>>
>>> --Kit
>>>
>>> On Wed, Jun 24, 2020 at 11:13 AM Javier Buzzi 
>>> wrote:
>>>
 I looked at the libs that do what we want:

 django-configurations - it looks like they use environment variables /
 either via loading them from the environ or a key/value pair file. Having
 classes inside the settings.py might be weird to people.. at the least very
 different.
 confucius - very simplistic, only supports environ and is classed
 based, similar to django-configurations.
 django-environ - supports env file and environ, non-class based.
 dynaconf - supports all kinds of loading options (toml, json, ini,
 environ,