Re: #21978 include production-ready web server

['Tobias McNulty' via Django developers (Contributions to Django itself)]

At Florian's suggestion in another thread, I propose that we move further 
discussion of the settings refactor to the forum:  
https://forum.djangoproject.com/t/settings-refactor/17352

On Wednesday, November 2, 2022 at 2:58:29 PM UTC-4 f.apo...@gmail.com wrote:

> The speckenv example highlights the problems when using `literal_eval`: 
> A=[] will parse as list but A=[a] will parse as string again (if I read the 
> code correctly). This ambiguity makes it really hard to explain. When using 
> `literal_eval` one imo has to be consistent and don't fall back to a str 
> and require strings to be written as A='[]' (with quotes). Then again this 
> is rather ugly and hard to get right/explain if you try to pass that in via 
> a shell ala `A='[]' django-admin runserver` and now A is a list again :/
>
> Cheers,
> Florian
>
> On Wednesday, November 2, 2022 at 8:20:00 AM UTC+1 m...@feinheit.ch wrote:
>
>> On Tue, Nov 1, 2022 at 10:34 AM Florian Apolloner  
>> wrote:
>>
>>> Right, that would work. I am wondering though if we want to go all in on 
>>> a typed config object like that or in a first step only have a simple API 
>>> like `config.get('DEBUG', cast=bool)`.
>>>
>>>
>> I found a neat trick in a 12factor library somewhere (I don't remember 
>> where unfortunately) of using ast.literal_eval for config variables:
>> https://docs.python.org/3/library/ast.html#ast.literal_eval
>>
>> https://github.com/matthiask/speckenv/blob/94722bcc95b94a9e6b1a1aada2bd977c43bfb767/speckenv.py#L63
>>
>> This may be too cute or magical for Django. It certainly works well and 
>> avoids having to be overspecific with types. I find it boring to tell the 
>> config loader that True is a bool and "hello" is a string. That being said 
>> I don't really have a strong preference here apart from having a strong 
>> dislike for the "several settings files" pattern. It's much more obvious to 
>> me if e.g. `MIDDLEWARE` is configured in a single location and not modified 
>> in other files. 
>>
>> Thanks,
>> Matthias
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3c783c4b-13ec-4e23-9fe1-101b92e4c49bn%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

The speckenv example highlights the problems when using `literal_eval`: 
A=[] will parse as list but A=[a] will parse as string again (if I read the 
code correctly). This ambiguity makes it really hard to explain. When using 
`literal_eval` one imo has to be consistent and don't fall back to a str 
and require strings to be written as A='[]' (with quotes). Then again this 
is rather ugly and hard to get right/explain if you try to pass that in via 
a shell ala `A='[]' django-admin runserver` and now A is a list again :/

Cheers,
Florian

On Wednesday, November 2, 2022 at 8:20:00 AM UTC+1 m...@feinheit.ch wrote:

> On Tue, Nov 1, 2022 at 10:34 AM Florian Apolloner  
> wrote:
>
>> Right, that would work. I am wondering though if we want to go all in on 
>> a typed config object like that or in a first step only have a simple API 
>> like `config.get('DEBUG', cast=bool)`.
>>
>>
> I found a neat trick in a 12factor library somewhere (I don't remember 
> where unfortunately) of using ast.literal_eval for config variables:
> https://docs.python.org/3/library/ast.html#ast.literal_eval
>
> https://github.com/matthiask/speckenv/blob/94722bcc95b94a9e6b1a1aada2bd977c43bfb767/speckenv.py#L63
>
> This may be too cute or magical for Django. It certainly works well and 
> avoids having to be overspecific with types. I find it boring to tell the 
> config loader that True is a bool and "hello" is a string. That being said 
> I don't really have a strong preference here apart from having a strong 
> dislike for the "several settings files" pattern. It's much more obvious to 
> me if e.g. `MIDDLEWARE` is configured in a single location and not modified 
> in other files. 
>
> Thanks,
> Matthias
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ad38315d-2a80-49de-9aa3-a93ec4ca567an%40googlegroups.com.

Re: #21978 include production-ready web server

[Matthias Kestenholz]

On Tue, Nov 1, 2022 at 10:34 AM Florian Apolloner 
wrote:

> Right, that would work. I am wondering though if we want to go all in on a
> typed config object like that or in a first step only have a simple API
> like `config.get('DEBUG', cast=bool)`.
>
>
I found a neat trick in a 12factor library somewhere (I don't remember
where unfortunately) of using ast.literal_eval for config variables:
https://docs.python.org/3/library/ast.html#ast.literal_eval
https://github.com/matthiask/speckenv/blob/94722bcc95b94a9e6b1a1aada2bd977c43bfb767/speckenv.py#L63

This may be too cute or magical for Django. It certainly works well and
avoids having to be overspecific with types. I find it boring to tell the
config loader that True is a bool and "hello" is a string. That being said
I don't really have a strong preference here apart from having a strong
dislike for the "several settings files" pattern. It's much more obvious to
me if e.g. `MIDDLEWARE` is configured in a single location and not modified
in other files.

Thanks,
Matthias

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANvPqgCTFEoO%2BG8-UNmHRPckto0DPoJb5-d596oexAc2kqxtNQ%40mail.gmail.com.

Re: #21978 include production-ready web server

['st...@jigsawtech.co.uk' via Django developers (Contributions to Django itself)]

config.get is how python-decouple works which is what I use for config and 
deployment settings.

On Tuesday, 1 November 2022 at 09:34:15 UTC f.apo...@gmail.com wrote:

> Right, that would work. I am wondering though if we want to go all in on a 
> typed config object like that or in a first step only have a simple API 
> like `config.get('DEBUG', cast=bool)`.
>
> On Tuesday, November 1, 2022 at 3:35:46 AM UTC+1 pe...@lincolnloop.com 
> wrote:
>
>> On Mon, Oct 31, 2022 at 11:52 AM Florian Apolloner  
>> wrote:
>>
>>> On Monday, October 31, 2022 at 5:27:02 PM UTC+1 pe...@lincolnloop.com 
>>> wrote:
>>>
 In my ideal scenario, the default is a hard-coded settings file for the 
 project (deprecating DJANGO_SETTINGS_MODULE env var) and we have 
 CONFIG_LOADERS defined in the settings that could do env, toml, etc. 
 Perhaps things like django.setup, 

>>> django.core.wsgi.get_wsgi_application, and 
 django.core.management.execute_from_command_line could accept the settings 
 module as an argument? django-admin could accept a --settings flag?

>>>
>>> I doubt that will ever fly. There is no project so to say in Django. 
>>> Currently literally everything depends on knowing a 
>>> `DJANGO_SETTINGS_MODULE` from which everything else can follow (even the 
>>> wsgi app…). So without a settings file how would you know that hard-coded 
>>> path for it? Django has no concept of a project that you could import. You 
>>> also cannot easily put `CONFIG_LOADERS` in a settings file like we 
>>> currently have because then how are you going to evaluate that while also 
>>> executing all the other module level stuff. And django-admin command all 
>>> already support a `--settings` flag (as alternative to 
>>> DJANGO_SETTINGS_MODULE ;))
>>>
>>> Either way I doubt we will deprecating DJANGO_SETTINGS_MODULE any time 
>>> soon. And I don't see it such of a problem with being there. Projects can 
>>> set it (and startproject already does this) to a fixed value in manage.py & 
>>> wsgi.py. For a project that installs a custom entrypoint script, you can 
>>> also preset it to whatever you like.
>>>
>>> Please note that we are trying to load stuff from files & env vars in 
>>> the first step, not throw out settings.py at the same time…
>>>
>>
>> I figured it wouldn't fly :)
>>
>> I misunderstood your chicken-egg question above. What I've been doing is 
>> defining a "Config" class and instantiating it in the settings file. 
>> Instantiation of the class (or calling an explicit method on it) would do 
>> the env/file loading and the instance becomes your `env` object. Rather 
>> than having a CONFIG_LOADERS setting, you could define them on the Config 
>> class or when you instantiate it. This is pretty similar to how goodconf 
>> works today. https://github.com/lincolnloop/goodconf#quick-start
>>  
>>
>>>
>>> Cheers,
>>> Florian
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>>
>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5ebb70e7-9118-4989-9a32-7b1faa9f032dn%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

Right, that would work. I am wondering though if we want to go all in on a 
typed config object like that or in a first step only have a simple API 
like `config.get('DEBUG', cast=bool)`.

On Tuesday, November 1, 2022 at 3:35:46 AM UTC+1 pe...@lincolnloop.com 
wrote:

> On Mon, Oct 31, 2022 at 11:52 AM Florian Apolloner  
> wrote:
>
>> On Monday, October 31, 2022 at 5:27:02 PM UTC+1 pe...@lincolnloop.com 
>> wrote:
>>
>>> In my ideal scenario, the default is a hard-coded settings file for the 
>>> project (deprecating DJANGO_SETTINGS_MODULE env var) and we have 
>>> CONFIG_LOADERS defined in the settings that could do env, toml, etc. 
>>> Perhaps things like django.setup, 
>>>
>> django.core.wsgi.get_wsgi_application, and 
>>> django.core.management.execute_from_command_line could accept the settings 
>>> module as an argument? django-admin could accept a --settings flag?
>>>
>>
>> I doubt that will ever fly. There is no project so to say in Django. 
>> Currently literally everything depends on knowing a 
>> `DJANGO_SETTINGS_MODULE` from which everything else can follow (even the 
>> wsgi app…). So without a settings file how would you know that hard-coded 
>> path for it? Django has no concept of a project that you could import. You 
>> also cannot easily put `CONFIG_LOADERS` in a settings file like we 
>> currently have because then how are you going to evaluate that while also 
>> executing all the other module level stuff. And django-admin command all 
>> already support a `--settings` flag (as alternative to 
>> DJANGO_SETTINGS_MODULE ;))
>>
>> Either way I doubt we will deprecating DJANGO_SETTINGS_MODULE any time 
>> soon. And I don't see it such of a problem with being there. Projects can 
>> set it (and startproject already does this) to a fixed value in manage.py & 
>> wsgi.py. For a project that installs a custom entrypoint script, you can 
>> also preset it to whatever you like.
>>
>> Please note that we are trying to load stuff from files & env vars in the 
>> first step, not throw out settings.py at the same time…
>>
>
> I figured it wouldn't fly :)
>
> I misunderstood your chicken-egg question above. What I've been doing is 
> defining a "Config" class and instantiating it in the settings file. 
> Instantiation of the class (or calling an explicit method on it) would do 
> the env/file loading and the instance becomes your `env` object. Rather 
> than having a CONFIG_LOADERS setting, you could define them on the Config 
> class or when you instantiate it. This is pretty similar to how goodconf 
> works today. https://github.com/lincolnloop/goodconf#quick-start
>  
>
>>
>> Cheers,
>> Florian
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bab31aa7-5351-488b-9b6e-b7264da7c711n%40googlegroups.com.

Re: #21978 include production-ready web server

[Peter Baumgartner]

On Mon, Oct 31, 2022 at 11:52 AM Florian Apolloner 
wrote:

> On Monday, October 31, 2022 at 5:27:02 PM UTC+1 pe...@lincolnloop.com
> wrote:
>
>> In my ideal scenario, the default is a hard-coded settings file for the
>> project (deprecating DJANGO_SETTINGS_MODULE env var) and we have
>> CONFIG_LOADERS defined in the settings that could do env, toml, etc.
>> Perhaps things like django.setup,
>>
> django.core.wsgi.get_wsgi_application, and
>> django.core.management.execute_from_command_line could accept the settings
>> module as an argument? django-admin could accept a --settings flag?
>>
>
> I doubt that will ever fly. There is no project so to say in Django.
> Currently literally everything depends on knowing a
> `DJANGO_SETTINGS_MODULE` from which everything else can follow (even the
> wsgi app…). So without a settings file how would you know that hard-coded
> path for it? Django has no concept of a project that you could import. You
> also cannot easily put `CONFIG_LOADERS` in a settings file like we
> currently have because then how are you going to evaluate that while also
> executing all the other module level stuff. And django-admin command all
> already support a `--settings` flag (as alternative to
> DJANGO_SETTINGS_MODULE ;))
>
> Either way I doubt we will deprecating DJANGO_SETTINGS_MODULE any time
> soon. And I don't see it such of a problem with being there. Projects can
> set it (and startproject already does this) to a fixed value in manage.py &
> wsgi.py. For a project that installs a custom entrypoint script, you can
> also preset it to whatever you like.
>
> Please note that we are trying to load stuff from files & env vars in the
> first step, not throw out settings.py at the same time…
>

I figured it wouldn't fly :)

I misunderstood your chicken-egg question above. What I've been doing is
defining a "Config" class and instantiating it in the settings file.
Instantiation of the class (or calling an explicit method on it) would do
the env/file loading and the instance becomes your `env` object. Rather
than having a CONFIG_LOADERS setting, you could define them on the Config
class or when you instantiate it. This is pretty similar to how goodconf
works today. https://github.com/lincolnloop/goodconf#quick-start


>
> Cheers,
> Florian
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6K9znR0RwrW%2B9SVMOy56mg%3DOuAYafBExGJf8n-TFdefhhKeA%40mail.gmail.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

On Monday, October 31, 2022 at 5:27:02 PM UTC+1 pe...@lincolnloop.com wrote:

> In my ideal scenario, the default is a hard-coded settings file for the 
> project (deprecating DJANGO_SETTINGS_MODULE env var) and we have 
> CONFIG_LOADERS defined in the settings that could do env, toml, etc. 
> Perhaps things like django.setup, 
>
django.core.wsgi.get_wsgi_application, and 
> django.core.management.execute_from_command_line could accept the settings 
> module as an argument? django-admin could accept a --settings flag?
>

I doubt that will ever fly. There is no project so to say in Django. 
Currently literally everything depends on knowing a 
`DJANGO_SETTINGS_MODULE` from which everything else can follow (even the 
wsgi app…). So without a settings file how would you know that hard-coded 
path for it? Django has no concept of a project that you could import. You 
also cannot easily put `CONFIG_LOADERS` in a settings file like we 
currently have because then how are you going to evaluate that while also 
executing all the other module level stuff. And django-admin command all 
already support a `--settings` flag (as alternative to 
DJANGO_SETTINGS_MODULE ;))

Either way I doubt we will deprecating DJANGO_SETTINGS_MODULE any time 
soon. And I don't see it such of a problem with being there. Projects can 
set it (and startproject already does this) to a fixed value in manage.py & 
wsgi.py. For a project that installs a custom entrypoint script, you can 
also preset it to whatever you like.

Please note that we are trying to load stuff from files & env vars in the 
first step, not throw out settings.py at the same time…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b0cc8e42-0781-4a69-b8a1-12cc6929c2f7n%40googlegroups.com.

Re: #21978 include production-ready web server

[Peter Baumgartner]

On Sun, Oct 30, 2022 at 7:57 AM Florian Apolloner 
wrote:

> Hi Peter,
>
> On Saturday, October 29, 2022 at 6:07:16 PM UTC+2 pe...@lincolnloop.com
> wrote:
>
>> Since you asked for feedback, I'll tell you what I'd do differently (and
>>> this is also a sign that production environment vary widely).
>>>
>>
>> Production environments do vary widely, but I don't think that should
>> stop us from providing defaults that are one-size-fits-most. startproject
>> already does this and users are welcome to adjust it how they see fit.
>>
>
> Agreed, I should have written this better. I expect there to be plenty of
> different opinions on this topic and it might be hard to figure out the
> one-size-fits-most without being to limiting.
>
>
>> * As soon as health checks check something aside from simply returning a
>>> 200 status code I'd want to be able to limit access to them
>>>
>>
>> Yep, that seems reasonable. Although depending on the level of failure,
>> you may not be able to log in to see them. Starting with just a view that
>> responds with a 200 is reasonable.
>>
>
> Sure and with limiting access I was more thinking about source IPs than
> actual logins (ie open health endpoint to the internal network from which
> the loadbalancer connects)
>
>
>>  * I do hate `DJANGO_ENV` but I guess there is no way around it.
>>> Currently my projects often piggy-back on DEBUG but that can be rather hard
>>> from time to time
>>>
>>
>> I'm open to other suggestions here, but am kind of in the same boat.
>> There may be times when you briefly want DEBUG=True in a non-public
>> deployed environment or DEBUG=False in a development environment.
>> Overloading that makes those scenarios a challenge.
>>
>
> Jupp, that is also the reason why I am in a love/hate relationship with it
> :)
>
>
>>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env
>>> variables that leak down the whole process-tree. django-environ does
>>> support this; that said there are other options like dynaconf which also
>>> support hashicorp vault which is a massive plus imo. Just being able to
>>> utilize vault for free from every django application out there would be
>>> stellar.
>>>
>>
>> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew
>> about dynaconf which does a lot of the same things. I would love to see
>> Django adopt a split between "settings" (which I see as things that don't
>> change across environments) and "configuration" (things that you set per
>> environment). I did switch to django-environ's FileAwareEnv [2] which
>> allows loading from files, but not in the yaml./toml sense. I didn't go all
>> in here because I felt like it was a more controversial stance and at the
>> end of the day, you probably still need to support environment variables
>> because many (most?) PaaS providers don't have support for files like that.
>>
>
> Yes, being able to override stuff via env variables should probably be
> always allowed. And as long as we do have a "schema" of some kind (be that
> via config.get_bool or typing annotation on a configuration object) it is
> also relatively easy to convert env variables to their correct form (ie
> boolean).
>
> With the goal of getting something merged into Django eventually, I felt
>> like environment variables were the lowest bar. We have precedence for
>> loading from environment variables with DJANGO_SETTINGS_MODULE. Do you
>> think a more full-featured library like dynaconf would make it harder to
>> push some of this upstream? Users that want to use dynaconf or goodconf or
>> whatever can still edit their settings and use those like they do today.
>>
>
> Yes and no. A 3rd party dependency will make it certainly harder to get
> this merged upstream. On the other hand I wouldn't want to support env
> variables only in the first iteration if that makes adding support for
> "properly" typed schemes (yaml/toml) harder in the future. From a viability
> point I'd like to see that we have an API that can handle hierarchical
> typed config and env variables. If we get that right the reasons to use
> something else will be rather small I guess.
>
> > Had another thought here... what about providing a pluggable
> configuration backend? Django could ship with some native backends like
> environment variables and toml (when it's available). Third party packages
> could provide Vault, secret files, etc. Same as how storages currently
> works and how you can easily drop in something like django-storages.
>
> Yes, the main question though is how the api contract would look like and
> how we are going to "solve" the chicken egg problem of how those backends
> are activated? We cannot really put `SETTING_LOADERS =
> ["django.settings.loaders.env", "django.settings.loader.toml"]` into a
> `settings.py` file and then use `config.get("DATABASE_URL")` just below to
> use the DATABASE_URL. So like `DJANGO_SETTINGS_MODULE` this would be
> something that has to be in an env var before Djang

Re: #21978 include production-ready web server

[Peter Baumgartner]

Hey Tobias! Not using DJANGO_SETTINGS_MODULE is due to my personal dislike
of it :)
I don't like the idea of a totally different Python file getting imported
based on an environment variable and I feel like having different Python
files for each environment isn't a great pattern. I think it is easier to
have major deviations between environments that aren't ever tested until
that environment is reached. I don't love Python being the language for
this type of configuration. I think something static like toml/yaml (or env
vars in a pinch) is easier to reason about, easier to dynamically generate,
and ultimately needs to be used if you want to avoid hard-coding secrets in
your repo.

All that being said, I realize the difference is subtle and goes against
the grain of what's already there. I'm willing to go whatever route makes
it easiest to get accepted.

On Sat, Oct 29, 2022 at 6:02 PM 'Tobias McNulty' via Django developers
(Contributions to Django itself)  wrote:

> Hi Pete,
>
> A pluggable backend sounds like a good idea.
>
> Regarding DJANGO_ENV, why not ship the template with two settings files
> and use the existing DJANGO_SETTINGS_MODULE, instead of adding a new
> environment variable?
>
> Maybe that would be controversial too, I'm not sure, but perhaps less so
> than introducing a new paradigm? In any event, perhaps a pluggable backend
> would allow everyone to be happy, or very nearly so... 😅
>
> To be clear, I'm fully in support of this, no matter how it ends up.
> Providing a decent production-capable server and some sane
> settings/configuration defaults for people to work from -- rather than
> having to scour the documentation for every mention of a production
> environment and figure out how to make those changes by hand -- will be a
> big win IMO. As it stands, first time users might reasonably assume they
> should use Apache to deploy to production (maybe they should, I'm not
> sure...but I haven't seen it done in over a decade):
>
> "We’ve included this with Django so you can develop things rapidly,
> without having to deal with configuring a production server – such as
> Apache – until you’re ready for production."
>
> https://docs.djangoproject.com/en/4.1/intro/tutorial01/
>
> Cheers,
> Tobias
>
> On Sat, Oct 29, 2022, 7:15 PM Peter Baumgartner 
> wrote:
>
>>
>>
>> On Sat, Oct 29, 2022 at 10:06 AM Peter Baumgartner 
>> wrote:
>>
>>> Thanks for the thorough review Florian! Some comments inline...
>>>
>>> On Wed, Oct 26, 2022 at 1:30 AM Florian Apolloner 
>>> wrote:
>>>
 Hi Pete,

 this does look interesting. And I agree that this is something that
 would be nice to have in core in one form or another. That said I think we
 have to do it iteratively (ie in smaller parts) and evaluate along the way.

>>>
>>> Agreed. I think adding the additional settings with some sort of
>>> production switch is the least obtrusive place to start.
>>>
>>>

 Since you asked for feedback, I'll tell you what I'd do differently
 (and this is also a sign that production environment vary widely).

>>>
>>> Production environments do vary widely, but I don't think that should
>>> stop us from providing defaults that are one-size-fits-most. startproject
>>> already does this and users are welcome to adjust it how they see fit.
>>>
>>>

  * As soon as health checks check something aside from simply returning
 a 200 status code I'd want to be able to limit access to them

>>>
>>> Yep, that seems reasonable. Although depending on the level of failure,
>>> you may not be able to log in to see them. Starting with just a view that
>>> responds with a 200 is reasonable.
>>>
>>>
  * I like my dev env to be as close as possible to production so I
 usually also use whitenoise in dev. I had to many times where dev worked
 and whitenoise in prod failed then :D

>>>
>>> Also a good point... I'll add that.
>>>
>>>
  * I do hate `DJANGO_ENV` but I guess there is no way around it.
 Currently my projects often piggy-back on DEBUG but that can be rather hard
 from time to time

>>>
>>> I'm open to other suggestions here, but am kind of in the same boat.
>>> There may be times when you briefly want DEBUG=True in a non-public
>>> deployed environment or DEBUG=False in a development environment.
>>> Overloading that makes those scenarios a challenge.
>>>
>>>
  * I prefer settings in settings files (ini/yaml/toml) as opposed to
 env variables that leak down the whole process-tree. django-environ does
 support this; that said there are other options like dynaconf which also
 support hashicorp vault which is a massive plus imo. Just being able to
 utilize vault for free from every django application out there would be
 stellar.

>>>
>>> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew
>>> about dynaconf which does a lot of the same things. I would love to see
>>> Django adopt a split between "settings" (

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Tobias,

On Sunday, October 30, 2022 at 2:02:36 AM UTC+2 tob...@caktusgroup.com 
wrote:

> Regarding DJANGO_ENV, why not ship the template with two settings files 
> and use the existing DJANGO_SETTINGS_MODULE, instead of adding a new 
> environment variable?
>

That is certainly one way to do it and probably the most controversial 
point. Ie what does DJANGO_ENV=dev|staging|prod offer over 
`DJANGO_SETTINGS_MODULE=your_project.settings.dev|staging|prod`

Maybe that would be controversial too, I'm not sure, but perhaps less so 
> than introducing a new paradigm? In any event, perhaps a pluggable backend 
> would allow everyone to be happy, or very nearly so... 😅
>

If we can come up with a reasonable API :) I wonder if a setuptools 
entrypoint to automatically enable your backend solely by installing it 
alongside the product would be one way to allow for easy customization… We 
might need two things to get this all started: An documented API contract 
that we can discuss and also a set of usecases that we do want to allow?

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1e0a0540-fcfa-45b0-bedc-f213aab9b267n%40googlegroups.com.

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Peter,

On Saturday, October 29, 2022 at 6:07:16 PM UTC+2 pe...@lincolnloop.com 
wrote:

> Since you asked for feedback, I'll tell you what I'd do differently (and 
>> this is also a sign that production environment vary widely).
>>
>
> Production environments do vary widely, but I don't think that should stop 
> us from providing defaults that are one-size-fits-most. startproject 
> already does this and users are welcome to adjust it how they see fit.
>

Agreed, I should have written this better. I expect there to be plenty of 
different opinions on this topic and it might be hard to figure out the 
one-size-fits-most without being to limiting.
 

> * As soon as health checks check something aside from simply returning a 
>> 200 status code I'd want to be able to limit access to them
>>
>
> Yep, that seems reasonable. Although depending on the level of failure, 
> you may not be able to log in to see them. Starting with just a view that 
> responds with a 200 is reasonable.
>

Sure and with limiting access I was more thinking about source IPs than 
actual logins (ie open health endpoint to the internal network from which 
the loadbalancer connects)
  

>  * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently 
>> my projects often piggy-back on DEBUG but that can be rather hard from time 
>> to time
>>
>
> I'm open to other suggestions here, but am kind of in the same boat. There 
> may be times when you briefly want DEBUG=True in a non-public deployed 
> environment or DEBUG=False in a development environment. Overloading that 
> makes those scenarios a challenge.
>

Jupp, that is also the reason why I am in a love/hate relationship with it 
:)
 

>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env 
>> variables that leak down the whole process-tree. django-environ does 
>> support this; that said there are other options like dynaconf which also 
>> support hashicorp vault which is a massive plus imo. Just being able to 
>> utilize vault for free from every django application out there would be 
>> stellar.
>>
>
> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew 
> about dynaconf which does a lot of the same things. I would love to see 
> Django adopt a split between "settings" (which I see as things that don't 
> change across environments) and "configuration" (things that you set per 
> environment). I did switch to django-environ's FileAwareEnv [2] which 
> allows loading from files, but not in the yaml./toml sense. I didn't go all 
> in here because I felt like it was a more controversial stance and at the 
> end of the day, you probably still need to support environment variables 
> because many (most?) PaaS providers don't have support for files like that.
>

Yes, being able to override stuff via env variables should probably be 
always allowed. And as long as we do have a "schema" of some kind (be that 
via config.get_bool or typing annotation on a configuration object) it is 
also relatively easy to convert env variables to their correct form (ie 
boolean).

With the goal of getting something merged into Django eventually, I felt 
> like environment variables were the lowest bar. We have precedence for 
> loading from environment variables with DJANGO_SETTINGS_MODULE. Do you 
> think a more full-featured library like dynaconf would make it harder to 
> push some of this upstream? Users that want to use dynaconf or goodconf or 
> whatever can still edit their settings and use those like they do today.
>

Yes and no. A 3rd party dependency will make it certainly harder to get 
this merged upstream. On the other hand I wouldn't want to support env 
variables only in the first iteration if that makes adding support for 
"properly" typed schemes (yaml/toml) harder in the future. From a viability 
point I'd like to see that we have an API that can handle hierarchical 
typed config and env variables. If we get that right the reasons to use 
something else will be rather small I guess. 
 
> Had another thought here... what about providing a pluggable 
configuration backend? Django could ship with some native backends like 
environment variables and toml (when it's available). Third party packages 
could provide Vault, secret files, etc. Same as how storages currently 
works and how you can easily drop in something like django-storages.

Yes, the main question though is how the api contract would look like and 
how we are going to "solve" the chicken egg problem of how those backends 
are activated? We cannot really put `SETTING_LOADERS = 
["django.settings.loaders.env", "django.settings.loader.toml"]` into a 
`settings.py` file and then use `config.get("DATABASE_URL")` just below to 
use the DATABASE_URL. So like `DJANGO_SETTINGS_MODULE` this would be 
something that has to be in an env var before Django starts. But I guess 
that would mostly be okay and projects might be able to pass their own 
loaders to `django.setup()` or configure 

Re: #21978 include production-ready web server

['Tobias McNulty' via Django developers (Contributions to Django itself)]

Hi Pete,

A pluggable backend sounds like a good idea.

Regarding DJANGO_ENV, why not ship the template with two settings files and
use the existing DJANGO_SETTINGS_MODULE, instead of adding a new
environment variable?

Maybe that would be controversial too, I'm not sure, but perhaps less so
than introducing a new paradigm? In any event, perhaps a pluggable backend
would allow everyone to be happy, or very nearly so... 😅

To be clear, I'm fully in support of this, no matter how it ends up.
Providing a decent production-capable server and some sane
settings/configuration defaults for people to work from -- rather than
having to scour the documentation for every mention of a production
environment and figure out how to make those changes by hand -- will be a
big win IMO. As it stands, first time users might reasonably assume they
should use Apache to deploy to production (maybe they should, I'm not
sure...but I haven't seen it done in over a decade):

"We’ve included this with Django so you can develop things rapidly, without
having to deal with configuring a production server – such as Apache –
until you’re ready for production."

https://docs.djangoproject.com/en/4.1/intro/tutorial01/

Cheers,
Tobias

On Sat, Oct 29, 2022, 7:15 PM Peter Baumgartner 
wrote:

>
>
> On Sat, Oct 29, 2022 at 10:06 AM Peter Baumgartner 
> wrote:
>
>> Thanks for the thorough review Florian! Some comments inline...
>>
>> On Wed, Oct 26, 2022 at 1:30 AM Florian Apolloner 
>> wrote:
>>
>>> Hi Pete,
>>>
>>> this does look interesting. And I agree that this is something that
>>> would be nice to have in core in one form or another. That said I think we
>>> have to do it iteratively (ie in smaller parts) and evaluate along the way.
>>>
>>
>> Agreed. I think adding the additional settings with some sort of
>> production switch is the least obtrusive place to start.
>>
>>
>>>
>>> Since you asked for feedback, I'll tell you what I'd do differently (and
>>> this is also a sign that production environment vary widely).
>>>
>>
>> Production environments do vary widely, but I don't think that should
>> stop us from providing defaults that are one-size-fits-most. startproject
>> already does this and users are welcome to adjust it how they see fit.
>>
>>
>>>
>>>  * As soon as health checks check something aside from simply returning
>>> a 200 status code I'd want to be able to limit access to them
>>>
>>
>> Yep, that seems reasonable. Although depending on the level of failure,
>> you may not be able to log in to see them. Starting with just a view that
>> responds with a 200 is reasonable.
>>
>>
>>>  * I like my dev env to be as close as possible to production so I
>>> usually also use whitenoise in dev. I had to many times where dev worked
>>> and whitenoise in prod failed then :D
>>>
>>
>> Also a good point... I'll add that.
>>
>>
>>>  * I do hate `DJANGO_ENV` but I guess there is no way around it.
>>> Currently my projects often piggy-back on DEBUG but that can be rather hard
>>> from time to time
>>>
>>
>> I'm open to other suggestions here, but am kind of in the same boat.
>> There may be times when you briefly want DEBUG=True in a non-public
>> deployed environment or DEBUG=False in a development environment.
>> Overloading that makes those scenarios a challenge.
>>
>>
>>>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env
>>> variables that leak down the whole process-tree. django-environ does
>>> support this; that said there are other options like dynaconf which also
>>> support hashicorp vault which is a massive plus imo. Just being able to
>>> utilize vault for free from every django application out there would be
>>> stellar.
>>>
>>
>> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew
>> about dynaconf which does a lot of the same things. I would love to see
>> Django adopt a split between "settings" (which I see as things that don't
>> change across environments) and "configuration" (things that you set per
>> environment). I did switch to django-environ's FileAwareEnv [2] which
>> allows loading from files, but not in the yaml./toml sense. I didn't go all
>> in here because I felt like it was a more controversial stance and at the
>> end of the day, you probably still need to support environment variables
>> because many (most?) PaaS providers don't have support for files like that.
>>
>> With the goal of getting something merged into Django eventually, I felt
>> like environment variables were the lowest bar. We have precedence for
>> loading from environment variables with DJANGO_SETTINGS_MODULE. Do you
>> think a more full-featured library like dynaconf would make it harder to
>> push some of this upstream? Users that want to use dynaconf or goodconf or
>> whatever can still edit their settings and use those like they do today.
>>
>> [1] https://github.com/lincolnloop/goodconf
>> [2]
>> https://django-environ.readthedocs.io/en/latest/tips.html#docker-style-file-based-variables
>>
>

Re: #21978 include production-ready web server

[Peter Baumgartner]

On Sat, Oct 29, 2022 at 10:06 AM Peter Baumgartner 
wrote:

> Thanks for the thorough review Florian! Some comments inline...
>
> On Wed, Oct 26, 2022 at 1:30 AM Florian Apolloner 
> wrote:
>
>> Hi Pete,
>>
>> this does look interesting. And I agree that this is something that would
>> be nice to have in core in one form or another. That said I think we have
>> to do it iteratively (ie in smaller parts) and evaluate along the way.
>>
>
> Agreed. I think adding the additional settings with some sort of
> production switch is the least obtrusive place to start.
>
>
>>
>> Since you asked for feedback, I'll tell you what I'd do differently (and
>> this is also a sign that production environment vary widely).
>>
>
> Production environments do vary widely, but I don't think that should stop
> us from providing defaults that are one-size-fits-most. startproject
> already does this and users are welcome to adjust it how they see fit.
>
>
>>
>>  * As soon as health checks check something aside from simply returning a
>> 200 status code I'd want to be able to limit access to them
>>
>
> Yep, that seems reasonable. Although depending on the level of failure,
> you may not be able to log in to see them. Starting with just a view that
> responds with a 200 is reasonable.
>
>
>>  * I like my dev env to be as close as possible to production so I
>> usually also use whitenoise in dev. I had to many times where dev worked
>> and whitenoise in prod failed then :D
>>
>
> Also a good point... I'll add that.
>
>
>>  * I do hate `DJANGO_ENV` but I guess there is no way around it.
>> Currently my projects often piggy-back on DEBUG but that can be rather hard
>> from time to time
>>
>
> I'm open to other suggestions here, but am kind of in the same boat. There
> may be times when you briefly want DEBUG=True in a non-public deployed
> environment or DEBUG=False in a development environment. Overloading that
> makes those scenarios a challenge.
>
>
>>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env
>> variables that leak down the whole process-tree. django-environ does
>> support this; that said there are other options like dynaconf which also
>> support hashicorp vault which is a massive plus imo. Just being able to
>> utilize vault for free from every django application out there would be
>> stellar.
>>
>
> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew
> about dynaconf which does a lot of the same things. I would love to see
> Django adopt a split between "settings" (which I see as things that don't
> change across environments) and "configuration" (things that you set per
> environment). I did switch to django-environ's FileAwareEnv [2] which
> allows loading from files, but not in the yaml./toml sense. I didn't go all
> in here because I felt like it was a more controversial stance and at the
> end of the day, you probably still need to support environment variables
> because many (most?) PaaS providers don't have support for files like that.
>
> With the goal of getting something merged into Django eventually, I felt
> like environment variables were the lowest bar. We have precedence for
> loading from environment variables with DJANGO_SETTINGS_MODULE. Do you
> think a more full-featured library like dynaconf would make it harder to
> push some of this upstream? Users that want to use dynaconf or goodconf or
> whatever can still edit their settings and use those like they do today.
>
> [1] https://github.com/lincolnloop/goodconf
> [2]
> https://django-environ.readthedocs.io/en/latest/tips.html#docker-style-file-based-variables
>

Had another thought here... what about providing a pluggable configuration
backend? Django could ship with some native backends like environment
variables and toml (when it's available). Third party packages could
provide Vault, secret files, etc. Same as how storages currently works and
how you can easily drop in something like django-storages.


>
>
>>
>> Another thing about settings, and this is something I really like in
>> spring-boot is their relaxed binding for configuration variables. You can
>> look at
>> https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0
>> for an example. They describe there how hierarchical variables are handled
>> and translated between yaml and env variables. Of course this binding is
>> not exactly bidirectional always (I think) but it allows for a few nice
>> things. First off you can start with a config file in yaml (one might
>> dislike yaml but I find it easier to read than say a .env file with 100
>> settings):
>>
>> ```
>> db:
>>   host: myhost
>>   port: 1234
>>
>> caches:
>>   default:
>>  type: memory
>>   second:
>> type: redis
>> ```
>>
>> which would translate to the following env variables:
>> ```
>> DB_HOST: myhost
>> DB_PORT: 1234
>> CACHES_DEFAULT_TYPE: memory
>> CACHES_SECOND_TYPE: redis
>> ```
>>
>> This is imo really a powerful concept and allows for so much nicer and
>> 

Re: #21978 include production-ready web server

[Peter Baumgartner]

You don't get typing out-of-the-box with straight environment variables.
Handling things like booleans can be tricky. You probably want to handle
"True", "true", "1", "yes", etc. Naive implementations can do the wrong
thing silently and introduce a major security hole (accidentally using
DEBUG=True). There are also security concerns with environment variables
but in some cases they are the only option you have, I summarized a few of
the problems here: https://github.com/lincolnloop/goodconf#why

On Wed, Oct 26, 2022 at 2:26 AM Jure Erznožnik 
wrote:

> IMHO, adding YAML support would introduce a dependency that's not needed:
> why not just use Python, instead of ini/yaml/toml?
>
> Projects could have a (.gitignore) env.py that settings.py imports and
> then sets the appropriate variables from it. You can use straight
> assignment, e.g.
>
> LANGUAGE_CODE = env.LANGUAGE_CODE
>
> or a getattr for optional settings:
>
> LANGUAGE_CODE = getattr(env, 'LANGUAGE_CODE', 'en-gb')
>
> No dependencies at all, not even internal python ones.
>
> But maybe I'm just being ignorant: there must be good reasons if core
> python accepted toml. In that case, I hope to be enlightened (no sarcasm
> here).
>
> LP,
> Jure
>
> On 26. 10. 22 09:30, Florian Apolloner wrote:
>
> Hi Pete,
>
> this does look interesting. And I agree that this is something that would
> be nice to have in core in one form or another. That said I think we have
> to do it iteratively (ie in smaller parts) and evaluate along the way.
>
> Since you asked for feedback, I'll tell you what I'd do differently (and
> this is also a sign that production environment vary widely).
>
>  * As soon as health checks check something aside from simply returning a
> 200 status code I'd want to be able to limit access to them
>  * I like my dev env to be as close as possible to production so I usually
> also use whitenoise in dev. I had to many times where dev worked and
> whitenoise in prod failed then :D
>  * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently
> my projects often piggy-back on DEBUG but that can be rather hard from time
> to time
>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env
> variables that leak down the whole process-tree. django-environ does
> support this; that said there are other options like dynaconf which also
> support hashicorp vault which is a massive plus imo. Just being able to
> utilize vault for free from every django application out there would be
> stellar.
>
> Another thing about settings, and this is something I really like in
> spring-boot is their relaxed binding for configuration variables. You can
> look at
> https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0
> for an example. They describe there how hierarchical variables are handled
> and translated between yaml and env variables. Of course this binding is
> not exactly bidirectional always (I think) but it allows for a few nice
> things. First off you can start with a config file in yaml (one might
> dislike yaml but I find it easier to read than say a .env file with 100
> settings):
>
> ```
> db:
>   host: myhost
>   port: 1234
>
> caches:
>   default:
>  type: memory
>   second:
> type: redis
> ```
>
> which would translate to the following env variables:
> ```
> DB_HOST: myhost
> DB_PORT: 1234
> CACHES_DEFAULT_TYPE: memory
> CACHES_SECOND_TYPE: redis
> ```
>
> This is imo really a powerful concept and allows for so much nicer and
> easier definitions.
>
> Cheers,
> Florian
>
> On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com
> wrote:
>
>> I put together a first stab at this at
>> https://github.com/lincolnloop/django-production. Feedback is
>> appreciated!
>>
>> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com <
>> pe...@lincolnloop.com> wrote:
>>
>>> I hadn't seen django-simple-deploy. At first glance, it looks
>>> interesting, but more platform-specific than what I was thinking.
>>>
>>> I'll play around with creating a third-party app that encapsulates some
>>> of these thoughts and see where I end up.
>>>
>>> Thanks for your time and feedback!
>>>
>>> On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com
>>> wrote:
>>>
 Hey Pete.

 Yes, there can be a lot of steps... (and trouble is every one has a
 different opinion once you get to specifics :)

 django-webserver looks good — in a similar ballpark to what I had in
 mind, yes, but I'd probably not want to bundle the options for each server
 myself, as that seems a lot to maintain over time... (Better to outsource
 to the individual projects no?)

 Have you seen what Eric Matthes is doing with django-simple-deploy?

 https://github.com/ehmatthes/django-simple-deploy

 I really like the idea: in an opinionated way, it applies the project
 changes you need to deploy (and it's plugable so folks could provide a
 *template* for their own flavour o

Re: #21978 include production-ready web server

[Peter Baumgartner]

Thanks for the thorough review Florian! Some comments inline...

On Wed, Oct 26, 2022 at 1:30 AM Florian Apolloner 
wrote:

> Hi Pete,
>
> this does look interesting. And I agree that this is something that would
> be nice to have in core in one form or another. That said I think we have
> to do it iteratively (ie in smaller parts) and evaluate along the way.
>

Agreed. I think adding the additional settings with some sort of production
switch is the least obtrusive place to start.


>
> Since you asked for feedback, I'll tell you what I'd do differently (and
> this is also a sign that production environment vary widely).
>

Production environments do vary widely, but I don't think that should stop
us from providing defaults that are one-size-fits-most. startproject
already does this and users are welcome to adjust it how they see fit.


>
>  * As soon as health checks check something aside from simply returning a
> 200 status code I'd want to be able to limit access to them
>

Yep, that seems reasonable. Although depending on the level of failure, you
may not be able to log in to see them. Starting with just a view that
responds with a 200 is reasonable.


>  * I like my dev env to be as close as possible to production so I usually
> also use whitenoise in dev. I had to many times where dev worked and
> whitenoise in prod failed then :D
>

Also a good point... I'll add that.


>  * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently
> my projects often piggy-back on DEBUG but that can be rather hard from time
> to time
>

I'm open to other suggestions here, but am kind of in the same boat. There
may be times when you briefly want DEBUG=True in a non-public deployed
environment or DEBUG=False in a development environment. Overloading that
makes those scenarios a challenge.


>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env
> variables that leak down the whole process-tree. django-environ does
> support this; that said there are other options like dynaconf which also
> support hashicorp vault which is a massive plus imo. Just being able to
> utilize vault for free from every django application out there would be
> stellar.
>

I'm 100% in agreement here. I wrote goodconf [1] for this before I knew
about dynaconf which does a lot of the same things. I would love to see
Django adopt a split between "settings" (which I see as things that don't
change across environments) and "configuration" (things that you set per
environment). I did switch to django-environ's FileAwareEnv [2] which
allows loading from files, but not in the yaml./toml sense. I didn't go all
in here because I felt like it was a more controversial stance and at the
end of the day, you probably still need to support environment variables
because many (most?) PaaS providers don't have support for files like that.

With the goal of getting something merged into Django eventually, I felt
like environment variables were the lowest bar. We have precedence for
loading from environment variables with DJANGO_SETTINGS_MODULE. Do you
think a more full-featured library like dynaconf would make it harder to
push some of this upstream? Users that want to use dynaconf or goodconf or
whatever can still edit their settings and use those like they do today.

[1] https://github.com/lincolnloop/goodconf
[2]
https://django-environ.readthedocs.io/en/latest/tips.html#docker-style-file-based-variables


>
> Another thing about settings, and this is something I really like in
> spring-boot is their relaxed binding for configuration variables. You can
> look at
> https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0
> for an example. They describe there how hierarchical variables are handled
> and translated between yaml and env variables. Of course this binding is
> not exactly bidirectional always (I think) but it allows for a few nice
> things. First off you can start with a config file in yaml (one might
> dislike yaml but I find it easier to read than say a .env file with 100
> settings):
>
> ```
> db:
>   host: myhost
>   port: 1234
>
> caches:
>   default:
>  type: memory
>   second:
> type: redis
> ```
>
> which would translate to the following env variables:
> ```
> DB_HOST: myhost
> DB_PORT: 1234
> CACHES_DEFAULT_TYPE: memory
> CACHES_SECOND_TYPE: redis
> ```
>
> This is imo really a powerful concept and allows for so much nicer and
> easier definitions.
>
> Cheers,
> Florian
>
> On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com
> wrote:
>
>> I put together a first stab at this at
>> https://github.com/lincolnloop/django-production. Feedback is
>> appreciated!
>>
>> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com <
>> pe...@lincolnloop.com> wrote:
>>
>>> I hadn't seen django-simple-deploy. At first glance, it looks
>>> interesting, but more platform-specific than what I was thinking.
>>>
>>> I'll play around with creating a third-party app that encapsulates some
>>

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Jure,

My point was mostly about hieracrhy, not the exact format. That said on a 
technical level it is true that the same can achieved via python -- after 
all this is what settings.py is currently. But the hard truth is that many 
people prefer a established configuration file format (for some definition 
of established). Having executable files as config files can also be 
dangerous. Assume a software that reads `env.py` from `[CWD, ~/.config/app, 
/etc/app]`. Now if someone manages to convince you to run the software in a 
directory that they preconfigured with an `env.py` you will execute the 
code they want you to execute. This is part of the reason why stuff like 
`direnv` and others explicitly require you to allow a directory before 
setting env variables from `.envrc` for you (the most common offender 
obviously being someone cloning a git repo and in the case of direnv simply 
stepping into the directory executes code).

How many of the standard python tooling ala pip, flake8, black, pytest, 
tox, isort allow you to have a python file as config file? At some point 
not supporting one of the more common config file format (.ini,.yaml,.toml) 
feels like swimming against the current without any real benefit. So yes, a 
python file would suffice but I guess that battle is lost already.

Cheers,
Florian


On Wednesday, October 26, 2022 at 10:26:55 AM UTC+2 jure.er...@gmail.com 
wrote:

> IMHO, adding YAML support would introduce a dependency that's not needed: 
> why not just use Python, instead of ini/yaml/toml?
>
> Projects could have a (.gitignore) env.py that settings.py imports and 
> then sets the appropriate variables from it. You can use straight 
> assignment, e.g. 
>
> LANGUAGE_CODE = env.LANGUAGE_CODE
>
> or a getattr for optional settings:
>
> LANGUAGE_CODE = getattr(env, 'LANGUAGE_CODE', 'en-gb')
>
> No dependencies at all, not even internal python ones.
>
> But maybe I'm just being ignorant: there must be good reasons if core 
> python accepted toml. In that case, I hope to be enlightened (no sarcasm 
> here).
>
> LP,
> Jure
>
> On 26. 10. 22 09:30, Florian Apolloner wrote:
>
> Hi Pete,
>
> this does look interesting. And I agree that this is something that would 
> be nice to have in core in one form or another. That said I think we have 
> to do it iteratively (ie in smaller parts) and evaluate along the way.
>
> Since you asked for feedback, I'll tell you what I'd do differently (and 
> this is also a sign that production environment vary widely).
>
>  * As soon as health checks check something aside from simply returning a 
> 200 status code I'd want to be able to limit access to them
>  * I like my dev env to be as close as possible to production so I usually 
> also use whitenoise in dev. I had to many times where dev worked and 
> whitenoise in prod failed then :D
>  * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently 
> my projects often piggy-back on DEBUG but that can be rather hard from time 
> to time
>  * I prefer settings in settings files (ini/yaml/toml) as opposed to env 
> variables that leak down the whole process-tree. django-environ does 
> support this; that said there are other options like dynaconf which also 
> support hashicorp vault which is a massive plus imo. Just being able to 
> utilize vault for free from every django application out there would be 
> stellar.
>
> Another thing about settings, and this is something I really like in 
> spring-boot is their relaxed binding for configuration variables. You can 
> look at 
> https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 
> for an example. They describe there how hierarchical variables are handled 
> and translated between yaml and env variables. Of course this binding is 
> not exactly bidirectional always (I think) but it allows for a few nice 
> things. First off you can start with a config file in yaml (one might 
> dislike yaml but I find it easier to read than say a .env file with 100 
> settings):
>
> ```
> db:
>   host: myhost
>   port: 1234
>
> caches:
>   default:
>  type: memory
>   second:
> type: redis
> ```
>
> which would translate to the following env variables:
> ```
> DB_HOST: myhost
> DB_PORT: 1234
> CACHES_DEFAULT_TYPE: memory
> CACHES_SECOND_TYPE: redis
> ``` 
>
> This is imo really a powerful concept and allows for so much nicer and 
> easier definitions.
>
> Cheers,
> Florian
>
> On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com 
> wrote:
>
>> I put together a first stab at this at 
>> https://github.com/lincolnloop/django-production. Feedback is 
>> appreciated!
>>
>> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com <
>> pe...@lincolnloop.com> wrote:
>>
>>> I hadn't seen django-simple-deploy. At first glance, it looks 
>>> interesting, but more platform-specific than what I was thinking.
>>>
>>> I'll play around with creating a third-party app that encapsulates some 
>>> of these thoughts and see where I e

Re: #21978 include production-ready web server

[Jure Erznožnik]

IMHO, adding YAML support would introduce a dependency that's not 
needed: why not just use Python, instead of ini/yaml/toml?


Projects could have a (.gitignore) env.py that settings.py imports and 
then sets the appropriate variables from it. You can use straight 
assignment, e.g.


   LANGUAGE_CODE = env.LANGUAGE_CODE

or a getattr for optional settings:

   LANGUAGE_CODE = getattr(env, 'LANGUAGE_CODE', 'en-gb')

No dependencies at all, not even internal python ones.

But maybe I'm just being ignorant: there must be good reasons if core 
python accepted toml. In that case, I hope to be enlightened (no sarcasm 
here).


LP,
Jure

On 26. 10. 22 09:30, Florian Apolloner wrote:

Hi Pete,

this does look interesting. And I agree that this is something that 
would be nice to have in core in one form or another. That said I 
think we have to do it iteratively (ie in smaller parts) and evaluate 
along the way.


Since you asked for feedback, I'll tell you what I'd do differently 
(and this is also a sign that production environment vary widely).


 * As soon as health checks check something aside from simply 
returning a 200 status code I'd want to be able to limit access to them
 * I like my dev env to be as close as possible to production so I 
usually also use whitenoise in dev. I had to many times where dev 
worked and whitenoise in prod failed then :D
 * I do hate `DJANGO_ENV` but I guess there is no way around it. 
Currently my projects often piggy-back on DEBUG but that can be rather 
hard from time to time
 * I prefer settings in settings files (ini/yaml/toml) as opposed to 
env variables that leak down the whole process-tree. django-environ 
does support this; that said there are other options like dynaconf 
which also support hashicorp vault which is a massive plus imo. Just 
being able to utilize vault for free from every django application out 
there would be stellar.


Another thing about settings, and this is something I really like in 
spring-boot is their relaxed binding for configuration variables. You 
can look at 
https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 
for an example. They describe there how hierarchical variables are 
handled and translated between yaml and env variables. Of course this 
binding is not exactly bidirectional always (I think) but it allows 
for a few nice things. First off you can start with a config file in 
yaml (one might dislike yaml but I find it easier to read than say a 
.env file with 100 settings):


```
db:
  host: myhost
  port: 1234

caches:
  default:
     type: memory
  second:
    type: redis
```

which would translate to the following env variables:
```
DB_HOST: myhost
DB_PORT: 1234
CACHES_DEFAULT_TYPE: memory
CACHES_SECOND_TYPE: redis
```

This is imo really a powerful concept and allows for so much nicer and 
easier definitions.


Cheers,
Florian

On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com 
wrote:


I put together a first stab at this at
https://github.com/lincolnloop/django-production. Feedback is
appreciated!

On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com
 wrote:

I hadn't seen django-simple-deploy. At first glance, it looks
interesting, but more platform-specific than what I was thinking.

I'll play around with creating a third-party app that
encapsulates some of these thoughts and see where I end up.

Thanks for your time and feedback!

On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6
carlton...@gmail.com wrote:

Hey Pete.

Yes, there can be a lot of steps... (and trouble is every
one has a different opinion once you get to specifics :)

django-webserver looks good — in a similar ballpark to
what I had in mind, yes, but I'd probably not want to
bundle the options for each server myself, as that seems a
lot to maintain over time... (Better to outsource to the
individual projects no?)

Have you seen what Eric Matthes is doing with
django-simple-deploy?

https://github.com/ehmatthes/django-simple-deploy

I really like the idea: in an opinionated way, it applies
the project changes you need to deploy (and it's plugable
so folks could provide a *template* for their own flavour
of deployment if they wanted, which is kind of important
given the number of options in the landscape, even if you
only to *ship* a beginner option.)

Tim Allen also recently raised improving the Django
project template, which I think is related here. (That's
on my list too: async this QTR, then hoping to take on
Adam's proposal to modernise the Request object for the
end of year, then 🤞 swinging back here for "OK, what does
it look like, what can we do?" after that: there's 

Re: #21978 include production-ready web server

[Florian Apolloner]

Hi Pete,

this does look interesting. And I agree that this is something that would 
be nice to have in core in one form or another. That said I think we have 
to do it iteratively (ie in smaller parts) and evaluate along the way.

Since you asked for feedback, I'll tell you what I'd do differently (and 
this is also a sign that production environment vary widely).

 * As soon as health checks check something aside from simply returning a 
200 status code I'd want to be able to limit access to them
 * I like my dev env to be as close as possible to production so I usually 
also use whitenoise in dev. I had to many times where dev worked and 
whitenoise in prod failed then :D
 * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently 
my projects often piggy-back on DEBUG but that can be rather hard from time 
to time
 * I prefer settings in settings files (ini/yaml/toml) as opposed to env 
variables that leak down the whole process-tree. django-environ does 
support this; that said there are other options like dynaconf which also 
support hashicorp vault which is a massive plus imo. Just being able to 
utilize vault for free from every django application out there would be 
stellar.

Another thing about settings, and this is something I really like in 
spring-boot is their relaxed binding for configuration variables. You can 
look at 
https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 for 
an example. They describe there how hierarchical variables are handled and 
translated between yaml and env variables. Of course this binding is not 
exactly bidirectional always (I think) but it allows for a few nice things. 
First off you can start with a config file in yaml (one might dislike yaml 
but I find it easier to read than say a .env file with 100 settings):

```
db:
  host: myhost
  port: 1234

caches:
  default:
 type: memory
  second:
type: redis
```

which would translate to the following env variables:
```
DB_HOST: myhost
DB_PORT: 1234
CACHES_DEFAULT_TYPE: memory
CACHES_SECOND_TYPE: redis
``` 

This is imo really a powerful concept and allows for so much nicer and 
easier definitions.

Cheers,
Florian

On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com wrote:

> I put together a first stab at this at 
> https://github.com/lincolnloop/django-production. Feedback is appreciated!
>
> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com <
> pe...@lincolnloop.com> wrote:
>
>> I hadn't seen django-simple-deploy. At first glance, it looks 
>> interesting, but more platform-specific than what I was thinking.
>>
>> I'll play around with creating a third-party app that encapsulates some 
>> of these thoughts and see where I end up.
>>
>> Thanks for your time and feedback!
>>
>> On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com 
>> wrote:
>>
>>> Hey Pete. 
>>>
>>> Yes, there can be a lot of steps... (and trouble is every one has a 
>>> different opinion once you get to specifics :) 
>>>
>>> django-webserver looks good — in a similar ballpark to what I had in 
>>> mind, yes, but I'd probably not want to bundle the options for each server 
>>> myself, as that seems a lot to maintain over time... (Better to outsource 
>>> to the individual projects no?)  
>>>
>>> Have you seen what Eric Matthes is doing with django-simple-deploy? 
>>>
>>> https://github.com/ehmatthes/django-simple-deploy
>>>
>>> I really like the idea: in an opinionated way, it applies the project 
>>> changes you need to deploy (and it's plugable so folks could provide a 
>>> *template* for their own flavour of deployment if they wanted, which is 
>>> kind of important given the number of options in the landscape, even if you 
>>> only to *ship* a beginner option.) 
>>>
>>> Tim Allen also recently raised improving the Django project template, 
>>> which I think is related here. (That's on my list too: async this QTR, then 
>>> hoping to take on Adam's proposal to modernise the Request object for the 
>>> end of year, then 🤞 swinging back here for "OK, what does it look like, 
>>> what can we do?" after that: there's a bunch of related tickets around 
>>> #21978 that it would be very sweet if we could clear up...) 
>>>
>>> I think all these thoughts are really pursuable outside of core in the 
>>> very short run, even if the goal is to merge them — it's much easier to 
>>> experiment that way, and then say, "This worked". 
>>>
>>> Anyhow, other than a "I've been thinking along these lines", and a 
>>> generally positive emote, I don't have too much more concrete at this 
>>> stage. 
>>> 👍
>>> C.
>>>
>>> On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner  
>>> wrote:
>>>
 Thanks for the additional background Carlton! I like the idea of
 having some sort of protocol that folks can follow to make
 ready-to-use Django server packages (django-webserver [1] is in the
 same vein), but I'd strive for more. pip install + INSTALLED_APPS is
 great, but getting to

Re: #21978 include production-ready web server

[Peter Baumgartner]

I put together a first stab at this at
https://github.com/lincolnloop/django-production. Feedback is appreciated!

On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com 
wrote:

> I hadn't seen django-simple-deploy. At first glance, it looks interesting,
> but more platform-specific than what I was thinking.
>
> I'll play around with creating a third-party app that encapsulates some of
> these thoughts and see where I end up.
>
> Thanks for your time and feedback!
>
> On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com
> wrote:
>
>> Hey Pete.
>>
>> Yes, there can be a lot of steps... (and trouble is every one has a
>> different opinion once you get to specifics :)
>>
>> django-webserver looks good — in a similar ballpark to what I had in
>> mind, yes, but I'd probably not want to bundle the options for each server
>> myself, as that seems a lot to maintain over time... (Better to outsource
>> to the individual projects no?)
>>
>> Have you seen what Eric Matthes is doing with django-simple-deploy?
>>
>> https://github.com/ehmatthes/django-simple-deploy
>>
>> I really like the idea: in an opinionated way, it applies the project
>> changes you need to deploy (and it's plugable so folks could provide a
>> *template* for their own flavour of deployment if they wanted, which is
>> kind of important given the number of options in the landscape, even if you
>> only to *ship* a beginner option.)
>>
>> Tim Allen also recently raised improving the Django project template,
>> which I think is related here. (That's on my list too: async this QTR, then
>> hoping to take on Adam's proposal to modernise the Request object for the
>> end of year, then 🤞 swinging back here for "OK, what does it look like,
>> what can we do?" after that: there's a bunch of related tickets around
>> #21978 that it would be very sweet if we could clear up...)
>>
>> I think all these thoughts are really pursuable outside of core in the
>> very short run, even if the goal is to merge them — it's much easier to
>> experiment that way, and then say, "This worked".
>>
>> Anyhow, other than a "I've been thinking along these lines", and a
>> generally positive emote, I don't have too much more concrete at this
>> stage.
>> 👍
>> C.
>>
>> On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner 
>> wrote:
>>
>>> Thanks for the additional background Carlton! I like the idea of
>>> having some sort of protocol that folks can follow to make
>>> ready-to-use Django server packages (django-webserver [1] is in the
>>> same vein), but I'd strive for more. pip install + INSTALLED_APPS is
>>> great, but getting to production with that approach can feel like
>>> death by a thousand paper cuts. Do that with gunicorn, whitenoise,
>>> dj-database-url, etc. and it ends up being a laundry list of things
>>> that, in my experience, people rarely get right on the first try and
>>> just result in frustration.
>>>
>>> It seems like part of the challenge here is reaching parity with the
>>> current runserver command, but is that necessary? Running `runserver
>>> --dev` locally and getting dev features like autoreloading and then
>>> having `runserver` use a different server for a deployed app doesn't
>>> seem unreasonable.
>>>
>>> [1] https://github.com/lincolnloop/django-webserver
>>>
>>> On Tue, Aug 23, 2022 at 10:14 AM Carlton Gibson
>>>  wrote:
>>> >
>>> > Hey Pete.
>>> >
>>> > Yes... this is a good one.
>>> >
>>> > It's difficult to see how we can even have an ASGI runserver in core,
>>> simply because there's no SimpleServer equivalent as there is for WSGI, and
>>> it's not clear one is on the card. (HTTP is getting **more** complex, and I
>>> can't see  Python bringing such into the standard library.)
>>> > So we have to use a third-party dependency for that.
>>> >
>>> > For the Channels v4 update (in progress now) I'm moving the runserver
>>> command to the Daphne package. This was because folks wanted to use
>>> Channels without the Daphne/Twisted dependency.
>>> >
>>> > See: https://github.com/django/daphne/pull/429
>>> >
>>> > tl;dr — it came out pretty clean TBH. You pip install daphne, then add
>>> "daphne" at the top of INSTALLED_APPS. (There's a system check to make sure
>>> you're not fighting staticfiles.)
>>> >
>>> > On the back of that I don't see why servers (any servers) can't
>>> provide a runserver command (or a django-* mini-package to go with it) and
>>> offer what they offer in development as well as production. (Obviously that
>>> all needs writing but it's just a management command.)
>>> >
>>> > Related is strategising the reloader.
>>> https://code.djangoproject.com/ticket/30213
>>> >
>>> > We could then host *Protocols* — "Your runserver should do this",
>>> "Implement this for your reloader", and so on — and (maybe) (slowly)
>>> replace built-in options here with either optional dependencies (pip
>>> install django["daphne"]) or recommendations to get going. (pip install
>>> django["starter"] maybe). — I don't know — we seem a long w

Re: #21978 include production-ready web server

[pe...@lincolnloop.com]

I hadn't seen django-simple-deploy. At first glance, it looks interesting, 
but more platform-specific than what I was thinking.

I'll play around with creating a third-party app that encapsulates some of 
these thoughts and see where I end up.

Thanks for your time and feedback!

On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com wrote:

> Hey Pete. 
>
> Yes, there can be a lot of steps... (and trouble is every one has a 
> different opinion once you get to specifics :) 
>
> django-webserver looks good — in a similar ballpark to what I had in mind, 
> yes, but I'd probably not want to bundle the options for each server 
> myself, as that seems a lot to maintain over time... (Better to outsource 
> to the individual projects no?)  
>
> Have you seen what Eric Matthes is doing with django-simple-deploy? 
>
> https://github.com/ehmatthes/django-simple-deploy
>
> I really like the idea: in an opinionated way, it applies the project 
> changes you need to deploy (and it's plugable so folks could provide a 
> *template* for their own flavour of deployment if they wanted, which is 
> kind of important given the number of options in the landscape, even if you 
> only to *ship* a beginner option.) 
>
> Tim Allen also recently raised improving the Django project template, 
> which I think is related here. (That's on my list too: async this QTR, then 
> hoping to take on Adam's proposal to modernise the Request object for the 
> end of year, then 🤞 swinging back here for "OK, what does it look like, 
> what can we do?" after that: there's a bunch of related tickets around 
> #21978 that it would be very sweet if we could clear up...) 
>
> I think all these thoughts are really pursuable outside of core in the 
> very short run, even if the goal is to merge them — it's much easier to 
> experiment that way, and then say, "This worked". 
>
> Anyhow, other than a "I've been thinking along these lines", and a 
> generally positive emote, I don't have too much more concrete at this 
> stage. 
> 👍
> C.
>
> On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner  
> wrote:
>
>> Thanks for the additional background Carlton! I like the idea of
>> having some sort of protocol that folks can follow to make
>> ready-to-use Django server packages (django-webserver [1] is in the
>> same vein), but I'd strive for more. pip install + INSTALLED_APPS is
>> great, but getting to production with that approach can feel like
>> death by a thousand paper cuts. Do that with gunicorn, whitenoise,
>> dj-database-url, etc. and it ends up being a laundry list of things
>> that, in my experience, people rarely get right on the first try and
>> just result in frustration.
>>
>> It seems like part of the challenge here is reaching parity with the
>> current runserver command, but is that necessary? Running `runserver
>> --dev` locally and getting dev features like autoreloading and then
>> having `runserver` use a different server for a deployed app doesn't
>> seem unreasonable.
>>
>> [1] https://github.com/lincolnloop/django-webserver
>>
>> On Tue, Aug 23, 2022 at 10:14 AM Carlton Gibson
>>  wrote:
>> >
>> > Hey Pete.
>> >
>> > Yes... this is a good one.
>> >
>> > It's difficult to see how we can even have an ASGI runserver in core, 
>> simply because there's no SimpleServer equivalent as there is for WSGI, and 
>> it's not clear one is on the card. (HTTP is getting **more** complex, and I 
>> can't see  Python bringing such into the standard library.)
>> > So we have to use a third-party dependency for that.
>> >
>> > For the Channels v4 update (in progress now) I'm moving the runserver 
>> command to the Daphne package. This was because folks wanted to use 
>> Channels without the Daphne/Twisted dependency.
>> >
>> > See: https://github.com/django/daphne/pull/429
>> >
>> > tl;dr — it came out pretty clean TBH. You pip install daphne, then add 
>> "daphne" at the top of INSTALLED_APPS. (There's a system check to make sure 
>> you're not fighting staticfiles.)
>> >
>> > On the back of that I don't see why servers (any servers) can't provide 
>> a runserver command (or a django-* mini-package to go with it) and offer 
>> what they offer in development as well as production. (Obviously that all 
>> needs writing but it's just a management command.)
>> >
>> > Related is strategising the reloader. 
>> https://code.djangoproject.com/ticket/30213
>> >
>> > We could then host *Protocols* — "Your runserver should do this", 
>> "Implement this for your reloader", and so on — and (maybe) (slowly) 
>> replace built-in options here with either optional dependencies (pip 
>> install django["daphne"]) or recommendations to get going. (pip install 
>> django["starter"] maybe). — I don't know — we seem a long way from that. (I 
>> don't suppose wsgiref is going anywhere too soon is it?)
>> >
>> > But to cut back to your point, I don't know that we need to **include** 
>> a server — just make it easy to get going. These days pip install + a line 
>> in 

Re: #21978 include production-ready web server

[Carlton Gibson]

Hey Pete.

Yes, there can be a lot of steps... (and trouble is every one has a
different opinion once you get to specifics :)

django-webserver looks good — in a similar ballpark to what I had in mind,
yes, but I'd probably not want to bundle the options for each server
myself, as that seems a lot to maintain over time... (Better to outsource
to the individual projects no?)

Have you seen what Eric Matthes is doing with django-simple-deploy?

https://github.com/ehmatthes/django-simple-deploy

I really like the idea: in an opinionated way, it applies the project
changes you need to deploy (and it's plugable so folks could provide a
*template* for their own flavour of deployment if they wanted, which is
kind of important given the number of options in the landscape, even if you
only to *ship* a beginner option.)

Tim Allen also recently raised improving the Django project template, which
I think is related here. (That's on my list too: async this QTR, then
hoping to take on Adam's proposal to modernise the Request object for the
end of year, then 🤞 swinging back here for "OK, what does it look like,
what can we do?" after that: there's a bunch of related tickets around
#21978 that it would be very sweet if we could clear up...)

I think all these thoughts are really pursuable outside of core in the very
short run, even if the goal is to merge them — it's much easier to
experiment that way, and then say, "This worked".

Anyhow, other than a "I've been thinking along these lines", and a
generally positive emote, I don't have too much more concrete at this
stage.
👍
C.

On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner 
wrote:

> Thanks for the additional background Carlton! I like the idea of
> having some sort of protocol that folks can follow to make
> ready-to-use Django server packages (django-webserver [1] is in the
> same vein), but I'd strive for more. pip install + INSTALLED_APPS is
> great, but getting to production with that approach can feel like
> death by a thousand paper cuts. Do that with gunicorn, whitenoise,
> dj-database-url, etc. and it ends up being a laundry list of things
> that, in my experience, people rarely get right on the first try and
> just result in frustration.
>
> It seems like part of the challenge here is reaching parity with the
> current runserver command, but is that necessary? Running `runserver
> --dev` locally and getting dev features like autoreloading and then
> having `runserver` use a different server for a deployed app doesn't
> seem unreasonable.
>
> [1] https://github.com/lincolnloop/django-webserver
>
> On Tue, Aug 23, 2022 at 10:14 AM Carlton Gibson
>  wrote:
> >
> > Hey Pete.
> >
> > Yes... this is a good one.
> >
> > It's difficult to see how we can even have an ASGI runserver in core,
> simply because there's no SimpleServer equivalent as there is for WSGI, and
> it's not clear one is on the card. (HTTP is getting **more** complex, and I
> can't see  Python bringing such into the standard library.)
> > So we have to use a third-party dependency for that.
> >
> > For the Channels v4 update (in progress now) I'm moving the runserver
> command to the Daphne package. This was because folks wanted to use
> Channels without the Daphne/Twisted dependency.
> >
> > See: https://github.com/django/daphne/pull/429
> >
> > tl;dr — it came out pretty clean TBH. You pip install daphne, then add
> "daphne" at the top of INSTALLED_APPS. (There's a system check to make sure
> you're not fighting staticfiles.)
> >
> > On the back of that I don't see why servers (any servers) can't provide
> a runserver command (or a django-* mini-package to go with it) and offer
> what they offer in development as well as production. (Obviously that all
> needs writing but it's just a management command.)
> >
> > Related is strategising the reloader.
> https://code.djangoproject.com/ticket/30213
> >
> > We could then host *Protocols* — "Your runserver should do this",
> "Implement this for your reloader", and so on — and (maybe) (slowly)
> replace built-in options here with either optional dependencies (pip
> install django["daphne"]) or recommendations to get going. (pip install
> django["starter"] maybe). — I don't know — we seem a long way from that. (I
> don't suppose wsgiref is going anywhere too soon is it?)
> >
> > But to cut back to your point, I don't know that we need to **include**
> a server — just make it easy to get going. These days pip install + a line
> in INSTALLED_APPS doesn't seem too much to ask (I think.)
> >
> > Was it really 4 years ago Tom left that comment on #21978 😳 (We'd
> finally close this as wontfix: Django doesn't ship a webserver.)
> >
> > Does that fit in your view, or would that scenario not be good enough?
> >
> > Kind Regards,
> >
> > Carlton
> >
> > On Tue, 23 Aug 2022 at 16:56, Peter Baumgartner 
> wrote:
> >>
> >> Hi all! I'd like to re-open the discussion around
> >> https://code.djangoproject.com/ticket/21978
> >>
> >> As a "batteries included" framework

Re: #21978 include production-ready web server

[Peter Baumgartner]

Thanks for the additional background Carlton! I like the idea of
having some sort of protocol that folks can follow to make
ready-to-use Django server packages (django-webserver [1] is in the
same vein), but I'd strive for more. pip install + INSTALLED_APPS is
great, but getting to production with that approach can feel like
death by a thousand paper cuts. Do that with gunicorn, whitenoise,
dj-database-url, etc. and it ends up being a laundry list of things
that, in my experience, people rarely get right on the first try and
just result in frustration.

It seems like part of the challenge here is reaching parity with the
current runserver command, but is that necessary? Running `runserver
--dev` locally and getting dev features like autoreloading and then
having `runserver` use a different server for a deployed app doesn't
seem unreasonable.

[1] https://github.com/lincolnloop/django-webserver

On Tue, Aug 23, 2022 at 10:14 AM Carlton Gibson
 wrote:
>
> Hey Pete.
>
> Yes... this is a good one.
>
> It's difficult to see how we can even have an ASGI runserver in core, simply 
> because there's no SimpleServer equivalent as there is for WSGI, and it's not 
> clear one is on the card. (HTTP is getting **more** complex, and I can't see  
> Python bringing such into the standard library.)
> So we have to use a third-party dependency for that.
>
> For the Channels v4 update (in progress now) I'm moving the runserver command 
> to the Daphne package. This was because folks wanted to use Channels without 
> the Daphne/Twisted dependency.
>
> See: https://github.com/django/daphne/pull/429
>
> tl;dr — it came out pretty clean TBH. You pip install daphne, then add 
> "daphne" at the top of INSTALLED_APPS. (There's a system check to make sure 
> you're not fighting staticfiles.)
>
> On the back of that I don't see why servers (any servers) can't provide a 
> runserver command (or a django-* mini-package to go with it) and offer what 
> they offer in development as well as production. (Obviously that all needs 
> writing but it's just a management command.)
>
> Related is strategising the reloader. 
> https://code.djangoproject.com/ticket/30213
>
> We could then host *Protocols* — "Your runserver should do this", "Implement 
> this for your reloader", and so on — and (maybe) (slowly) replace built-in 
> options here with either optional dependencies (pip install django["daphne"]) 
> or recommendations to get going. (pip install django["starter"] maybe). — I 
> don't know — we seem a long way from that. (I don't suppose wsgiref is going 
> anywhere too soon is it?)
>
> But to cut back to your point, I don't know that we need to **include** a 
> server — just make it easy to get going. These days pip install + a line in 
> INSTALLED_APPS doesn't seem too much to ask (I think.)
>
> Was it really 4 years ago Tom left that comment on #21978 😳 (We'd finally 
> close this as wontfix: Django doesn't ship a webserver.)
>
> Does that fit in your view, or would that scenario not be good enough?
>
> Kind Regards,
>
> Carlton
>
> On Tue, 23 Aug 2022 at 16:56, Peter Baumgartner  wrote:
>>
>> Hi all! I'd like to re-open the discussion around
>> https://code.djangoproject.com/ticket/21978
>>
>> As a "batteries included" framework, Django kind of leaves people to
>> fend for themselves when it comes to deployment. This makes it harder
>> than necessary for a developer to go from works-on-my-laptop to
>> works-on-the-internet. The docs here are great, but also daunting to a
>> newcomer https://docs.djangoproject.com/en/4.1/howto/deployment/
>>
>> I'd love to see Django provide some sane defaults that people can use
>> to deploy a site without having to make a bunch of decisions they
>> don't totally understand.
>>
>> I have two thoughts on how it could be handled:
>>
>> 1. Similar to the template backends. You get Django's template system
>> by default, but you're welcome to swap in Jinja2. I don't believe
>> there is a one-size-fits-all webserver, but probably a
>> one-size-fits-most.
>> 2. Similar to cache/database backends. You define the backend you want
>> to use and Django makes some sane choices for you and allows you to
>> serve the site via some common `manage.py` command.
>>
>> For the first option, Gunicorn is a popular choice, but afaik it does
>> not work on Windows. Waitress [1] is an interesting option for WSGI
>> serving and daphne exists for ASGI. Whitenoise is a reasonable choice
>> for serving static files. It doesn't yet support ASGI, but there has
>> been some activity on that front [2].
>>
>> Thanks!
>>
>> [1] https://pypi.org/project/waitress/
>> [2] https://github.com/evansd/whitenoise/pull/359
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the we

Re: #21978 include production-ready web server

[Carlton Gibson]

Hey Pete.

Yes... this is a good one.

It's difficult to see how we can even have an ASGI runserver in core,
simply because there's no SimpleServer equivalent as there is for WSGI, and
it's not clear one is on the card. (HTTP is getting **more** complex, and I
can't see  Python bringing such into the standard library.)
So we have to use a third-party dependency for that.

For the Channels v4 update (in progress now) I'm moving the runserver
command to the Daphne package. This was because folks wanted to use
Channels without the Daphne/Twisted dependency.

See: https://github.com/django/daphne/pull/429

tl;dr — it came out pretty clean TBH. You pip install daphne, then add
"daphne" at the top of INSTALLED_APPS. (There's a system check to make sure
you're not fighting staticfiles.)

On the back of that I don't see why servers (any servers) can't provide a
runserver command (or a django-* mini-package to go with it) and offer what
they offer in development as well as production. (Obviously that all needs
writing but it's just a management command.)

Related is strategising the reloader.
https://code.djangoproject.com/ticket/30213

We could then host *Protocols* — "Your runserver should do this",
"Implement this for your reloader", and so on — and (maybe) (slowly)
replace built-in options here with either optional dependencies (pip
install django["daphne"]) or recommendations to get going. (pip install
django["starter"] maybe). — I don't know — we seem a long way from that. (I
don't suppose wsgiref is going anywhere too soon is it?)

But to cut back to your point, I don't know that we need to **include** a
server — just make it easy to get going. These days pip install + a line in
INSTALLED_APPS doesn't seem too much to ask (I think.)

Was it really 4 years ago Tom left that comment on #21978
 😳 (We'd finally close this
as wontfix: Django doesn't ship a webserver.)

Does that fit in your view, or would that scenario not be good enough?

Kind Regards,

Carlton

On Tue, 23 Aug 2022 at 16:56, Peter Baumgartner 
wrote:

> Hi all! I'd like to re-open the discussion around
> https://code.djangoproject.com/ticket/21978
>
> As a "batteries included" framework, Django kind of leaves people to
> fend for themselves when it comes to deployment. This makes it harder
> than necessary for a developer to go from works-on-my-laptop to
> works-on-the-internet. The docs here are great, but also daunting to a
> newcomer https://docs.djangoproject.com/en/4.1/howto/deployment/
>
> I'd love to see Django provide some sane defaults that people can use
> to deploy a site without having to make a bunch of decisions they
> don't totally understand.
>
> I have two thoughts on how it could be handled:
>
> 1. Similar to the template backends. You get Django's template system
> by default, but you're welcome to swap in Jinja2. I don't believe
> there is a one-size-fits-all webserver, but probably a
> one-size-fits-most.
> 2. Similar to cache/database backends. You define the backend you want
> to use and Django makes some sane choices for you and allows you to
> serve the site via some common `manage.py` command.
>
> For the first option, Gunicorn is a popular choice, but afaik it does
> not work on Windows. Waitress [1] is an interesting option for WSGI
> serving and daphne exists for ASGI. Whitenoise is a reasonable choice
> for serving static files. It doesn't yet support ASGI, but there has
> been some activity on that front [2].
>
> Thanks!
>
> [1] https://pypi.org/project/waitress/
> [2] https://github.com/evansd/whitenoise/pull/359
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAC6K9zk5PmcxYWXRdvco1pWXnO%2BHoYoYHf0pg5Mw%3DgmdefZArg%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJwKpyTSGPJW%2BmdUADW8eBYaXakesHG9H9dkSzhLxWnCpBdH6A%40mail.gmail.com.