Hi All, I hope you are all well.
I've been thinking about static assets over the past week or so following my email on widget media. I collated the past 6 year's worth of discussions into a single source, with relevant extracts and links to sources. I've also set out some options and given my thoughts (FWIW). I've not yet done any detailed planning as I think we need to agree general principles first. Sorry this is a bit long, may need to go get a cup of tea first :-) I hope this is helpful to aid discussion on this topic 🤞 *Summary* The below sets out this historical context for ticket #22298 <https://code.djangoproject.com/ticket/22298> (What to do with Widget Media) and the wider discussions this generated on how Django should manage assets. Whilst there are some detailed points below, these are to help set context and to aid discussion. This is a wide and deep topic and therefore I suggest at this stage we discuss the topic with an aim to agree the overall direction. The principle I suggest we discuss is: *In alignment with it's "batteries included" philosophy, should Django have an aim to improve the management of static assets? * In addition to seeking agreement to this broad principle I'd also like to encourage discussion on the topics which fall under this category. Here are some questions to aid a discussion, although I fully expect the conversation to be wider than these initial suggestions. Some items may be easier to seek agreement on than others; however, all will take time and effort of the community to agree the correct design and implementation. 1. Should Django have an improved way of managing assets within apps (something akin to #29586) 2. Should Django have the ability to compile and compress assets? 3. Should Django have Webpack support? 4. Should Django have improved support for JS frameworks? 5. Should Django have support for NPM? Based on evidence below my view is that there is enough support to improve Django's support in this area generally. Assuming this is agreed I think the a range of options are: - Provide improve awareness of static assets with Django core to help 3rd party packages - In addition, integrate an asset pipeline - In addition, integration with webpack, and support JS frameworks. My personal view is that option one is fairly cautious, there are already 3rd party packages which enable this functionality, and that the third one is too ambitious. I suspect most people using JS frameworks are doing it anyway irrespective of what is part of Django code. I think there the most value can be added is for sites with mostly HTML / CSS and, but would like some help compiling CSS and "sprinkling" a bit of JS here and there. I'd therefore suggest we start by revisiting work previously done by Claude to understand if this is the right foundation. Once this is complete this will enable phase two to integrate an asset pipeline / compressor into Django. I would suggest that this is via a separate package to start with under the Django umbrella. I suspect the pace of iteration will be too fast for Django core, and we would like it to be more stable before merging it. Would it be possible to use some of the existing work that the wider community has already developed and explore building on (or 'just' use) an existing 3rd party package? (I don't know how acceptable this is, or not) My personal recommendation is also that ticket #22298 is closed or marked as someday / maybe. Widget Media may be useful in the future, we don't know yet. I would not support the deprecation of this functionality without something better to replace it. I also feel that a rename would have little benefit but would cause disruption for those using it and numbers may be high given it's been in place for many years. If agreement can be sought on this then it will help other Media related tickets to be progressed (example : #21987). Below is an overview of various comments, on tickets, mailing list and pull requests over the past 6 years. *The ticket in question was on Widget Media, So what does Widget Media do?* It is used to load CSS or JS to widgets / forms (e.g. add <script> tags). This is useful when a certain widget can be enhanced by some extra JS. A calendar would be a classic example of this. By using the media class the JS for the calendar is only loaded on pages where the widget is present. *How long has been around? *Media classes have been in Django for a long time. They are included from at least version 1.0. *And why may we want to rename to static, what's the history here?* In [Django 1.3] `*django.contrib.staticfiles` *was added "to handle static files in a generic way". In [Django 1.4]( https://docs.djangoproject.com/en/3.0/releases/1.4/#django-contrib-admin) "Django's admin's static files also followed this convention by removing `ADMIN_MEDIA_PREFIX` and replaced it with `STATIC_URL`". So whilst most of the project moved away from using the phase 'media'; Widgets continues to use this phrase. This is inconsistent and _could_ be fixed. *Now you've mentioned it, do we even need it?* Whilst the ticket originally suggested a name chance should be considered the discussion on the ticket is much deeper. This is where the complexity of this ticket grows significantly with and a wide range of options is available. Some of the options mentioned are: - Let's deprecate Widget Media, it's not best practice - Tell people to use django-compressor / pipeline - Should these tools be integrated? - Should there be better support for front end JS frameworks - Should we have support for webpack - We should support NPM All of these items I think come under the question of "How should Django work with assets?" *Django ecosystem* Currently Django has a number of 3rd party packages which help to manage assets. A couple of options which have been mentioned in previous discussions on this topic are django-compressor (>2k github stars, >250k downloads p/month) and django-pipeline (>1k GitHub stars, >100k downloads p/month). These both augment the standard Django functionality to "Compresses linked and inline JavaScript or CSS into a single cached file." In addition there is also Django webpack loader (2k GitHub Stars, >300k downloads p/month) "Use webpack to generate your static bundles without django's staticfiles or opaque wrappers." Some of these packages were also mentioned in the discussion on the django forum under the "top 5 packages" thread [4], but they did not feature time and time again. So whilst they have reasonable use, they don't seem to feature at the top of a good proportion of people's lists. *What do other frameworks do? * Previous comments have been that rails includes the following from Carlton [1] > *The static files story is a little different. It seems to me we don't > tell the best story there. * > *Rails has two things which we could be envious of, even if we didn't want > to copy exactly:* > > ** The frontend framework integration that's already been mentioned. *** > The very easy "Ajax your form", with controllers (i.e. for us "generic > views") automatically handling ajax form submissions. * > *Both these features get users further quicker in these aspects than we > are able to offer. * For a more in depth overview of rails, this link was previously provided to the mailing list [5]. Further a technical write up was provided [1a] *Previous work *Whilst I'm only seeking discussion on principles at this stage (should Django have these features). I think it's useful to draw reference to previous discussions and attempts to make progress in this area, as I feel it is useful context. It came up again as another GSOC. However, it was felt that the proposal itself wasn't feasible and that it was to big and complex to achieve through GSOC. [3] It also got a mention on a separate GSOC proposal again in 2018, not particularly aligned to this topic but generated some relevant topics. . This one was a bit more complex, but did gain some comments on what may be useful for Django. Tom's post from this thread is particularly helpful [6]. *"I'm perhaps a bit biased but I would be very interested in anything that > can make JavaScript a real first class citizen in Django"* Claude has also attempted to add a base structure to Django. Ticket was closed as it was felt that developing this within Django itself was the wrong approach. Florian: *I am just a little bit worried about adding this without any concrete plan > on how 3rd party apps are going to use it. What speaks against trying this > outside of core like channels? (I'll happily put it under the django > umbrella, but core seems a little bit fast tracked to me).* (https://code.djangoproject.com/ticket/29586) (https://github.com/django/django/pull/10218) (https://groups.google.com/forum/#!topic/django-developers/KYmNnvwXDUI) *So does any one use Widget Media it anyway?* Opinions in on the ticket are mixed, some comments of having never used it vs some which think it is quite common. Examples : A ticket where a site used it extensively enough to encounter performance issues [#30563](https://code.djangoproject.com/ticket/30563). The comment suggested 194,016 media objects were being loaded but only 13 unique items. Wagtail makes use of it. [Wagtail sourcecode]( https://github.com/wagtail/wagtail/search?l=Python&p=1&q=.media) *References* [1] https://groups.google.com/d/msg/django-developers/KVAZkRCq9KU/AwnM5UbCGQAJ [1a] https://blog.yourlabs.org/posts/2019-02-26-django-js-research-report/ [3] https://groups.google.com/forum/#!searchin/django-developers/media$20class%7Csort:date/django-developers/XF7F__mETrM/ECa4JpF-wRgJ <https://groups.google.com/forum/#!searchin/django-developers/media%2420class%7Csort:date/django-developers/XF7F__mETrM/ECa4JpF-wRgJ> [4]https://forum.djangoproject.com/t/top-5-3rd-party-packages/391 [5]https://medium.com/@hpux/rails-5-1-loves-javascript-a1d84d5318b [6] https://groups.google.com/d/msg/django-developers/ldhY8sAQc-0/h2RngIeZCQAJ -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/bbfaffce-f6e2-4fea-b21c-34db7886d153%40googlegroups.com.
