Thanks for the thorough review Florian! Some comments inline... On Wed, Oct 26, 2022 at 1:30 AM Florian Apolloner <f.apollo...@gmail.com> wrote:
> Hi Pete, > > this does look interesting. And I agree that this is something that would > be nice to have in core in one form or another. That said I think we have > to do it iteratively (ie in smaller parts) and evaluate along the way. > Agreed. I think adding the additional settings with some sort of production switch is the least obtrusive place to start. > > Since you asked for feedback, I'll tell you what I'd do differently (and > this is also a sign that production environment vary widely). > Production environments do vary widely, but I don't think that should stop us from providing defaults that are one-size-fits-most. startproject already does this and users are welcome to adjust it how they see fit. > > * As soon as health checks check something aside from simply returning a > 200 status code I'd want to be able to limit access to them > Yep, that seems reasonable. Although depending on the level of failure, you may not be able to log in to see them. Starting with just a view that responds with a 200 is reasonable. > * I like my dev env to be as close as possible to production so I usually > also use whitenoise in dev. I had to many times where dev worked and > whitenoise in prod failed then :D > Also a good point... I'll add that. > * I do hate `DJANGO_ENV` but I guess there is no way around it. Currently > my projects often piggy-back on DEBUG but that can be rather hard from time > to time > I'm open to other suggestions here, but am kind of in the same boat. There may be times when you briefly want DEBUG=True in a non-public deployed environment or DEBUG=False in a development environment. Overloading that makes those scenarios a challenge. > * I prefer settings in settings files (ini/yaml/toml) as opposed to env > variables that leak down the whole process-tree. django-environ does > support this; that said there are other options like dynaconf which also > support hashicorp vault which is a massive plus imo. Just being able to > utilize vault for free from every django application out there would be > stellar. > I'm 100% in agreement here. I wrote goodconf [1] for this before I knew about dynaconf which does a lot of the same things. I would love to see Django adopt a split between "settings" (which I see as things that don't change across environments) and "configuration" (things that you set per environment). I did switch to django-environ's FileAwareEnv [2] which allows loading from files, but not in the yaml./toml sense. I didn't go all in here because I felt like it was a more controversial stance and at the end of the day, you probably still need to support environment variables because many (most?) PaaS providers don't have support for files like that. With the goal of getting something merged into Django eventually, I felt like environment variables were the lowest bar. We have precedence for loading from environment variables with DJANGO_SETTINGS_MODULE. Do you think a more full-featured library like dynaconf would make it harder to push some of this upstream? Users that want to use dynaconf or goodconf or whatever can still edit their settings and use those like they do today. [1] https://github.com/lincolnloop/goodconf [2] https://django-environ.readthedocs.io/en/latest/tips.html#docker-style-file-based-variables > > Another thing about settings, and this is something I really like in > spring-boot is their relaxed binding for configuration variables. You can > look at > https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 > for an example. They describe there how hierarchical variables are handled > and translated between yaml and env variables. Of course this binding is > not exactly bidirectional always (I think) but it allows for a few nice > things. First off you can start with a config file in yaml (one might > dislike yaml but I find it easier to read than say a .env file with 100 > settings): > > ``` > db: > host: myhost > port: 1234 > > caches: > default: > type: memory > second: > type: redis > ``` > > which would translate to the following env variables: > ``` > DB_HOST: myhost > DB_PORT: 1234 > CACHES_DEFAULT_TYPE: memory > CACHES_SECOND_TYPE: redis > ``` > > This is imo really a powerful concept and allows for so much nicer and > easier definitions. > > Cheers, > Florian > > On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com > wrote: > >> I put together a first stab at this at >> https://github.com/lincolnloop/django-production. Feedback is >> appreciated! >> >> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com < >> pe...@lincolnloop.com> wrote: >> >>> I hadn't seen django-simple-deploy. At first glance, it looks >>> interesting, but more platform-specific than what I was thinking. >>> >>> I'll play around with creating a third-party app that encapsulates some >>> of these thoughts and see where I end up. >>> >>> Thanks for your time and feedback! >>> >>> On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com >>> wrote: >>> >>>> Hey Pete. >>>> >>>> Yes, there can be a lot of steps... (and trouble is every one has a >>>> different opinion once you get to specifics :) >>>> >>>> django-webserver looks good β in a similar ballpark to what I had in >>>> mind, yes, but I'd probably not want to bundle the options for each server >>>> myself, as that seems a lot to maintain over time... (Better to outsource >>>> to the individual projects no?) >>>> >>>> Have you seen what Eric Matthes is doing with django-simple-deploy? >>>> >>>> https://github.com/ehmatthes/django-simple-deploy >>>> >>>> I really like the idea: in an opinionated way, it applies the project >>>> changes you need to deploy (and it's plugable so folks could provide a >>>> *template* for their own flavour of deployment if they wanted, which is >>>> kind of important given the number of options in the landscape, even if you >>>> only to *ship* a beginner option.) >>>> >>>> Tim Allen also recently raised improving the Django project template, >>>> which I think is related here. (That's on my list too: async this QTR, then >>>> hoping to take on Adam's proposal to modernise the Request object for the >>>> end of year, then π€ swinging back here for "OK, what does it look like, >>>> what can we do?" after that: there's a bunch of related tickets around >>>> #21978 that it would be very sweet if we could clear up...) >>>> >>>> I think all these thoughts are really pursuable outside of core in the >>>> very short run, even if the goal is to merge them β it's much easier to >>>> experiment that way, and then say, "This worked". >>>> >>>> Anyhow, other than a "I've been thinking along these lines", and a >>>> generally positive emote, I don't have too much more concrete at this >>>> stage. >>>> π >>>> C. >>>> >>>> On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner <pe...@lincolnloop.com> >>>> wrote: >>>> >>>>> Thanks for the additional background Carlton! I like the idea of >>>>> having some sort of protocol that folks can follow to make >>>>> ready-to-use Django server packages (django-webserver [1] is in the >>>>> same vein), but I'd strive for more. pip install + INSTALLED_APPS is >>>>> great, but getting to production with that approach can feel like >>>>> death by a thousand paper cuts. Do that with gunicorn, whitenoise, >>>>> dj-database-url, etc. and it ends up being a laundry list of things >>>>> that, in my experience, people rarely get right on the first try and >>>>> just result in frustration. >>>>> >>>>> It seems like part of the challenge here is reaching parity with the >>>>> current runserver command, but is that necessary? Running `runserver >>>>> --dev` locally and getting dev features like autoreloading and then >>>>> having `runserver` use a different server for a deployed app doesn't >>>>> seem unreasonable. >>>>> >>>>> [1] https://github.com/lincolnloop/django-webserver >>>>> >>>>> On Tue, Aug 23, 2022 at 10:14 AM Carlton Gibson >>>>> <carlton...@gmail.com> wrote: >>>>> > >>>>> > Hey Pete. >>>>> > >>>>> > Yes... this is a good one. >>>>> > >>>>> > It's difficult to see how we can even have an ASGI runserver in >>>>> core, simply because there's no SimpleServer equivalent as there is for >>>>> WSGI, and it's not clear one is on the card. (HTTP is getting **more** >>>>> complex, and I can't see Python bringing such into the standard library.) >>>>> > So we have to use a third-party dependency for that. >>>>> > >>>>> > For the Channels v4 update (in progress now) I'm moving the >>>>> runserver command to the Daphne package. This was because folks wanted to >>>>> use Channels without the Daphne/Twisted dependency. >>>>> > >>>>> > See: https://github.com/django/daphne/pull/429 >>>>> > >>>>> > tl;dr β it came out pretty clean TBH. You pip install daphne, then >>>>> add "daphne" at the top of INSTALLED_APPS. (There's a system check to make >>>>> sure you're not fighting staticfiles.) >>>>> > >>>>> > On the back of that I don't see why servers (any servers) can't >>>>> provide a runserver command (or a django-* mini-package to go with it) and >>>>> offer what they offer in development as well as production. (Obviously >>>>> that >>>>> all needs writing but it's just a management command.) >>>>> > >>>>> > Related is strategising the reloader. >>>>> https://code.djangoproject.com/ticket/30213 >>>>> > >>>>> > We could then host *Protocols* β "Your runserver should do this", >>>>> "Implement this for your reloader", and so on β and (maybe) (slowly) >>>>> replace built-in options here with either optional dependencies (pip >>>>> install django["daphne"]) or recommendations to get going. (pip install >>>>> django["starter"] maybe). β I don't know β we seem a long way from that. >>>>> (I >>>>> don't suppose wsgiref is going anywhere too soon is it?) >>>>> > >>>>> > But to cut back to your point, I don't know that we need to >>>>> **include** a server β just make it easy to get going. These days pip >>>>> install + a line in INSTALLED_APPS doesn't seem too much to ask (I think.) >>>>> > >>>>> > Was it really 4 years ago Tom left that comment on #21978 π³ (We'd >>>>> finally close this as wontfix: Django doesn't ship a webserver.) >>>>> > >>>>> > Does that fit in your view, or would that scenario not be good >>>>> enough? >>>>> > >>>>> > Kind Regards, >>>>> > >>>>> > Carlton >>>>> > >>>>> > On Tue, 23 Aug 2022 at 16:56, Peter Baumgartner < >>>>> pe...@lincolnloop.com> wrote: >>>>> >> >>>>> >> Hi all! I'd like to re-open the discussion around >>>>> >> https://code.djangoproject.com/ticket/21978 >>>>> >> >>>>> >> As a "batteries included" framework, Django kind of leaves people to >>>>> >> fend for themselves when it comes to deployment. This makes it >>>>> harder >>>>> >> than necessary for a developer to go from works-on-my-laptop to >>>>> >> works-on-the-internet. The docs here are great, but also daunting >>>>> to a >>>>> >> newcomer https://docs.djangoproject.com/en/4.1/howto/deployment/ >>>>> >> >>>>> >> I'd love to see Django provide some sane defaults that people can >>>>> use >>>>> >> to deploy a site without having to make a bunch of decisions they >>>>> >> don't totally understand. >>>>> >> >>>>> >> I have two thoughts on how it could be handled: >>>>> >> >>>>> >> 1. Similar to the template backends. You get Django's template >>>>> system >>>>> >> by default, but you're welcome to swap in Jinja2. I don't believe >>>>> >> there is a one-size-fits-all webserver, but probably a >>>>> >> one-size-fits-most. >>>>> >> 2. Similar to cache/database backends. You define the backend you >>>>> want >>>>> >> to use and Django makes some sane choices for you and allows you to >>>>> >> serve the site via some common `manage.py` command. >>>>> >> >>>>> >> For the first option, Gunicorn is a popular choice, but afaik it >>>>> does >>>>> >> not work on Windows. Waitress [1] is an interesting option for WSGI >>>>> >> serving and daphne exists for ASGI. Whitenoise is a reasonable >>>>> choice >>>>> >> for serving static files. It doesn't yet support ASGI, but there has >>>>> >> been some activity on that front [2]. >>>>> >> >>>>> >> Thanks! >>>>> >> >>>>> >> [1] https://pypi.org/project/waitress/ >>>>> >> [2] https://github.com/evansd/whitenoise/pull/359 >>>>> >> >>>>> >> -- >>>>> >> You received this message because you are subscribed to the Google >>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>> >> To unsubscribe from this group and stop receiving emails from it, >>>>> send an email to django-develop...@googlegroups.com. >>>>> >> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/django-developers/CAC6K9zk5PmcxYWXRdvco1pWXnO%2BHoYoYHf0pg5Mw%3DgmdefZArg%40mail.gmail.com >>>>> . >>>>> > >>>>> > -- >>>>> > You received this message because you are subscribed to the Google >>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>> send an email to django-develop...@googlegroups.com. >>>>> > To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/django-developers/CAJwKpyTSGPJW%2BmdUADW8eBYaXakesHG9H9dkSzhLxWnCpBdH6A%40mail.gmail.com >>>>> . >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to django-develop...@googlegroups.com. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/django-developers/CAC6K9zmVDKcpM%3Dc9tyhrp2_K_dzLH5S9zMJcZSXduF9NkcoCKA%40mail.gmail.com >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django developers (Contributions to Django itself)" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-develop...@googlegroups.com. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-developers/493f7f12-541c-4e12-ac25-0fd341ce2dc2n%40googlegroups.com >>> <https://groups.google.com/d/msgid/django-developers/493f7f12-541c-4e12-ac25-0fd341ce2dc2n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/c79db112-a965-4700-90b2-2bcdcd9d65a8n%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/c79db112-a965-4700-90b2-2bcdcd9d65a8n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAC6K9znnVPts8NnNmAevSJRScBp9SXWzQNvs5gc90suNPV-0hw%40mail.gmail.com.
