Subresource Integrity
Hi,
You may have come across the (relatively) new subresource integrity
(SRI)[1] security feature that is available in Chrome and Firefox (its
coming to Edge[2] and Safari[3]) - it enables a hash to be provided as an
attribute on a linked JavaScript or CSS tag that the browser can test
against once the file has downloaded to ensure that it hasn't been tampered
with. Its main use case is for when you are using a third party CDN for
your static files. I have been looking at how best to implement it in/with
Django and believe that incorporating it into the 'staticfiles' framework
may be the best we to go.
Currently 'ManifestStaticFilesStorage' and 'CachedStaticFilesStorage' store
a simple dict mapping file path to path was a hash in the filename for
cache busting. I would like to propose that this is changed to a mapping of
file path to a tuple of (filepath_with_hash, hash_algo, file_hash_base64).
This allows us to add the subresource integrity attribute to any linked
file that is managed by either of those storages.
I have mocked up how this could be done [4], the changes I made
to django/contrib/staticfiles/storage.py are here [5].
There is then a simple template tag that includes the SRI token:
{% static_sri_token 'path/to/file.js' %}
>>> sha256-ATJWBF9ZhO9FS8vr9wEhYO+W9vLtd3jRZVmEdC5EL0s=
or the whole attribute:
{% static_sri_attrs 'path/to/file.js' %}
>>> integrity="sha256-ATJWBF9ZhO9FS8vr9wEhYO+W9vLtd3jRZVmEdC5EL0s="
crossorigin="anonymous"
By incorporating this into Django rather than a third party app it would
ensure that everyone gains access to this new security feature for free -
it can be documented with the static files docs, encouraging its use and
help to secure more peoples projects.
Please note that my code is just a proof of principle, and I would be very
happy to hear if anyone has any thought on a better way of implementing the
feature.
[1]:
https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity
[2]:
https://developer.microsoft.com/en-us/microsoft-edge/platform/status/subresourceintegrity/
[3]: https://webkit.org/status/#feature-subresource-integrity
[4]: https://gist.github.com/samwillis/90b5391f9dafc0662ae2af7b8f827bd9
[5]:
https://gist.github.com/samwillis/90b5391f9dafc0662ae2af7b8f827bd9/revisions#diff-80f99275da9c9dbb59bb177e83b9f13e
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/26ac849d-d6d1-4e9e-aa14-d16b0552a43f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Removing and renaming Django's Python 2 related helpers
A difference is that a deprecation starting in 2.2 is in 2 releases (2.2, 2.3) while the deprecation starting in 3.0 is in 3 releases (3.0, 3.1, 3.2). On Wednesday, January 25, 2017 at 10:30:40 AM UTC-5, Aymeric Augustin wrote: > > Hello Tim, > > On 25 Jan 2017, at 16:11, Tim Graham > > wrote: > > - in Django 2.2 for removal in Django 3.1 (August 2020) > - in Django 3.0 for removal in Django 4.0 (Dec. 2021) > > > Either option seems fine to me. > > Are there advantages to starting the deprecation in a LTS (2.2) vs. in the > release that follows a LTS (3.0)? > > -- > Aymeric. > > > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/75ba8da6-dac2-4b3f-bf7a-b64c0732693e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Default custom reverse manager
Hi all, I have a use case where I need to use a custom reverse manager (Or Related Manager?). Docs can be found here: https://docs.djangoproject.com/en/1.10/topics/db/queries/#using-custom-reverse-manager I was wondering if a default custom reverse manager exists. So instead of having to explicitly define a manager for every usage: b.entry_set(manager='entries').all() You could set the manager when defining the relationship: class Entry(models.Model): #... blog = models.ForeignKey(Blog, manager=EntryManager()) If this behavior doesn't exist, I'd be happy to look into implementing it. Omer -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/72bc6f2d-c8ad-4e66-a3a6-69b0e2e30098%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Removing and renaming Django's Python 2 related helpers
Hello Tim, On 25 Jan 2017, at 16:11, Tim Graham wrote: > - in Django 2.2 for removal in Django 3.1 (August 2020) > - in Django 3.0 for removal in Django 4.0 (Dec. 2021) Either option seems fine to me. Are there advantages to starting the deprecation in a LTS (2.2) vs. in the release that follows a LTS (3.0)? -- Aymeric. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1097C141-7EB8-4DC3-AB26-167399682815%40polytechnique.org. For more options, visit https://groups.google.com/d/optout.
Re: Removing and renaming Django's Python 2 related helpers
If we go the deprecation route, when do you want to start those deprecations? If deprecation start in Django 2.0, the removal happens in Django 3.0 (December 2019). Do you think third-party apps will drop support for Python 2 by then? For removal at a later date, the deprecations could start: - in Django 2.2 for removal in Django 3.1 (August 2020) - in Django 3.0 for removal in Django 4.0 (Dec. 2021) (Sam, I don't see much advantage to your proposal about six, considering it's not adding any burden to Django.) On Tuesday, January 24, 2017 at 6:21:53 PM UTC-5, Sam Willis wrote: > > Hi, > > An alternative option with 'six' is to replace it with an alias of six > propper (not vendored), something like this: > > # django/utils/six.py > try: > from six import * > except ImportError: > raise ImportError(( > "`django.utils.six` is deprecated, install six from pypi " > "(https://pypi.python.org/pypi/six) instead. After installing > it will be available at " > "`django.utils.six` until Django V2.xx.")) > except: > import warnings > warnings.warn(( > "The `django.utils.six` alias is deprecated and will be > removed in Django V2.xx.", > DeprecationWarning, 2 > ) > > That way it removes the need for it to be maintained in core but ensures > that other Django apps that are supporting py2 and 3 don't break. > > Sam > > On Tuesday, January 24, 2017 at 11:53:01 AM UTC, Aymeric Augustin wrote: >> >> Hello, >> >> Django is earning a lot of goodwill from its well-defined deprecation >> policy. It was recently improved to allow pluggable apps to work without >> import shims and without deprecation warnings from one LTS to the next. I >> don’t know the exact details but I believe that's the intent. >> >> Combined with a thorough documentation of backwards-incompatible changes, >> it helps many developers put up with changes, even those they don’t >> understand, disagree with, or require significant work to adapt >> applications. >> >> For this reason, in the context of the transition from Python 2 to Python >> 3, which is very backwards incompatible for people who haven’t been able to >> anticipate it, I’m in favor of a generous application of the deprecation >> policy. >> >> I’m +1 on deprecating rather than removing utilities that were mentioned >> in the documentation, notably python_2_unicode_compatible, which was a >> public API until it got merged into six and we started using six’ version. >> >> I’m +0 on deprecating rather than removing modules that a developer of a >> pluggable app would reasonably use for Python 2 compatibility, such >> as django.utils.lru_cache or django.utils.six. >> >> I’d just remove less visible functions such as django.utils._os.abspathu, >> provided they clearly fall in the “private API” bucket. >> >> Best regards, >> >> -- >> Aymeric. >> >> >> >> On 21 Jan 2017, at 21:55, Tim Graham wrote: >> >> As we worked on removing Python 2 compatibility code from master this >> week [0], we collected a number of import shims and functions that are only >> needed for code that wants to support Python 2 [1]. >> >> So far there is django.utils.six, as well as some undocumented things: >> >>- django.utils.lru_cache >>- django.utils._os.abspathu, upath, npath >>- django.utils.decorators.available_attrs >>- django.utils.encoding.python_2_unicode_compatible >> >> I'm advocating to remove the undocumented things in Django 3.0 (released >> Dec. 2019) or later without a deprecation. By that time, I hope third-party >> apps won't support Python 2 either and so part of adding Django 3.0 >> compatibility will include formally dropping support for Python 2 (if not >> done already) and removing usage of this stuff (a fairly easy find/replace, >> hopefully). >> >> Others have advocated to deprecate all these things, but I don't see much >> advantage. If I were maintaining an app, I'd rather be able to use import >> shims without warnings until Python 2 is gone. What's your preference? >> >> A similar concern applies to django.utils.http's urlquote() and >> urlquote_plus() (documented) and urlunquote() and urlunquote_plus() >> (undocumented). Claude has a PR that deprecates them [4] which I think is >> fine considering they are half documented. >> >> A related issue is the naming of the force_text() / force_str() and >> smart_text() / smart_str() functions. Aymeric proposed a PR to change all >> force_text() >> usage in Django to force_str() (they function identically on Python 3) [2], >> however, I feel this might create some confusion. For example, when >> backporting to 1.11 and earlier, we'll have to change force_str() to >> force_text() for Python 2 compatibility. Third-party apps might also be >> confused on which function to use. Claude proposed a similar that changes >> ugettext() >> and ungettext() to gettext() / ngettext() [3]. Would it be less >> confusing
Re: [ANNOUNCE] Django 1.11 alpha 1 released
For me, testing involves updating tests of third-party apps and my own Django projects to run against the new version. On Wednesday, January 25, 2017 at 7:37:48 AM UTC-5, Vimarsh Chaturvedi wrote: > > Hey, > > I have recently started trying to contribute to Django. > Wanted to know how can one help in testing for the final release? > > Using the new features in small projects or is there a formal procedure > for testing? > > On Wednesday, January 18, 2017 at 6:48:18 AM UTC+5:30, Tim Graham wrote: >> >> We've made the first release on the way to Django's next major release, >> Django 1.11! With two and a half months until the scheduled final release, >> we'll need timely testing from the community to ensure an on-time and >> stable release. Check out the blog post: >> >> https://www.djangoproject.com/weblog/2017/jan/17/django-111-alpha-1/ >> > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/18fce465-2a77-4819-ac13-6600c3d945a8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: _mysql_exceptions.OperationalError: (1366, '') when running tests against MySQL
Try adding this to both entries in your DATABASES test settings:
'TEST': {
'CHARSET': 'utf8',
'COLLATION': 'utf8_general_ci',
},
On Wednesday, January 25, 2017 at 7:36:41 AM UTC-5, JA Robson wrote:
>
> hi, I'm on OSX (Sierra), am running a local MySQL, version 5.7.17, running
> django '2.0.dev20170122010200', and python mysqlclient 1.3.9 and 1.3.1 (I
> tried both versions).
>
> when I try to run the tests, it consistently fails with this stack trace
>
> Traceback (most recent call last):
> File
> "/Users/test/projects/django-hack/django/django/db/backends/utils.py", line
> 63, in execute
> return self.cursor.execute(sql, params)
> File
> "/Users/test/projects/django-hack/django/django/db/backends/mysql/base.py",
> line 88, in execute
> return self.cursor.execute(query, args)
> File
> "/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/cursors.py",
>
> line 221, in execute
> self.errorhandler(self, exc, value)
> File
> "/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/connections.py",
>
> line 38, in defaulterrorhandler
> raise errorclass(errorvalue)
> _mysql_exceptions.OperationalError: (1366, '')
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "./runtests.py", line 466, in
> options.exclude_tags,
> File "./runtests.py", line 270, in django_tests
> extra_tests=extra_tests,
> File "/Users/test/projects/django-hack/django/django/test/runner.py",
> line 601, in run_tests
> old_config = self.setup_databases()
> File "/Users/test/projects/django-hack/django/django/test/runner.py",
> line 546, in setup_databases
> self.parallel, **kwargs
> File "/Users/test/projects/django-hack/django/django/test/utils.py",
> line 180, in setup_databases
> serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE',
> True),
> File
> "/Users/test/projects/django-hack/django/django/db/backends/base/creation.py",
>
> line 68, in create_test_db
> run_syncdb=True,
> File
> "/Users/test/projects/django-hack/django/django/core/management/__init__.py",
> line 128, in call_command
> return command.execute(*args, **defaults)
> File
> "/Users/test/projects/django-hack/django/django/core/management/base.py",
> line 327, in execute
> output = self.handle(*args, **options)
> File
> "/Users/test/projects/django-hack/django/django/core/management/commands/migrate.py",
>
> line 224, in handle
> self.verbosity, self.interactive, connection.alias,
> apps=post_migrate_apps, plan=plan,
> File
> "/Users/test/projects/django-hack/django/django/core/management/sql.py",
> line 51, in emit_post_migrate_signal
> **kwargs
> File
> "/Users/test/projects/django-hack/django/django/dispatch/dispatcher.py",
> line 179, in send
> for receiver in self._live_receivers(sender)
> File
> "/Users/test/projects/django-hack/django/django/dispatch/dispatcher.py",
> line 179, in
> for receiver in self._live_receivers(sender)
> File
> "/Users/test/projects/django-hack/django/django/contrib/auth/management/__init__.py",
>
> line 79, in create_permissions
> Permission.objects.using(using).bulk_create(perms)
> File
> "/Users/test/projects/django-hack/django/django/db/models/query.py", line
> 438, in bulk_create
> ids = self._batched_insert(objs_without_pk, fields, batch_size)
> File
> "/Users/test/projects/django-hack/django/django/db/models/query.py", line
> 1079, in _batched_insert
> self._insert(item, fields=fields, using=self.db)
> File
> "/Users/test/projects/django-hack/django/django/db/models/query.py", line
> 1056, in _insert
> return query.get_compiler(using=using).execute_sql(return_id)
> File
> "/Users/test/projects/django-hack/django/django/db/models/sql/compiler.py",
> line 1094, in execute_sql
> cursor.execute(sql, params)
> File
> "/Users/test/projects/django-hack/django/django/db/backends/utils.py", line
> 63, in execute
> return self.cursor.execute(sql, params)
> File "/Users/test/projects/django-hack/django/django/db/utils.py", line
> 93, in __exit__
> six.reraise(dj_exc_type, dj_exc_value, traceback)
> File "/Users/test/projects/django-hack/django/django/utils/six.py", line
> 683, in reraise
> raise value.with_traceback(tb)
> File
> "/Users/test/projects/django-hack/django/django/db/backends/utils.py", line
> 63, in execute
> return self.cursor.execute(sql, params)
> File
> "/Users/test/projects/django-hack/django/django/db/backends/mysql/base.py",
> line 88, in execute
> return self.cursor.execute(query, args)
> File
> "/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/cursors.py",
>
> line 221, in execute
> self.errorhandler(self, exc, value)
> File
> "/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/connections.py",
>
> line 38, in defaulterrorhandler
> raise
_mysql_exceptions.OperationalError: (1366, '') when running tests against MySQL
hi, I'm on OSX (Sierra), am running a local MySQL, version 5.7.17, running
django '2.0.dev20170122010200', and python mysqlclient 1.3.9 and 1.3.1 (I
tried both versions).
when I try to run the tests, it consistently fails with this stack trace
Traceback (most recent call last):
File
"/Users/test/projects/django-hack/django/django/db/backends/utils.py", line
63, in execute
return self.cursor.execute(sql, params)
File
"/Users/test/projects/django-hack/django/django/db/backends/mysql/base.py",
line 88, in execute
return self.cursor.execute(query, args)
File
"/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/cursors.py",
line 221, in execute
self.errorhandler(self, exc, value)
File
"/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/connections.py",
line 38, in defaulterrorhandler
raise errorclass(errorvalue)
_mysql_exceptions.OperationalError: (1366, '')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "./runtests.py", line 466, in
options.exclude_tags,
File "./runtests.py", line 270, in django_tests
extra_tests=extra_tests,
File "/Users/test/projects/django-hack/django/django/test/runner.py",
line 601, in run_tests
old_config = self.setup_databases()
File "/Users/test/projects/django-hack/django/django/test/runner.py",
line 546, in setup_databases
self.parallel, **kwargs
File "/Users/test/projects/django-hack/django/django/test/utils.py", line
180, in setup_databases
serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE',
True),
File
"/Users/test/projects/django-hack/django/django/db/backends/base/creation.py",
line 68, in create_test_db
run_syncdb=True,
File
"/Users/test/projects/django-hack/django/django/core/management/__init__.py",
line 128, in call_command
return command.execute(*args, **defaults)
File
"/Users/test/projects/django-hack/django/django/core/management/base.py",
line 327, in execute
output = self.handle(*args, **options)
File
"/Users/test/projects/django-hack/django/django/core/management/commands/migrate.py",
line 224, in handle
self.verbosity, self.interactive, connection.alias,
apps=post_migrate_apps, plan=plan,
File
"/Users/test/projects/django-hack/django/django/core/management/sql.py",
line 51, in emit_post_migrate_signal
**kwargs
File
"/Users/test/projects/django-hack/django/django/dispatch/dispatcher.py",
line 179, in send
for receiver in self._live_receivers(sender)
File
"/Users/test/projects/django-hack/django/django/dispatch/dispatcher.py",
line 179, in
for receiver in self._live_receivers(sender)
File
"/Users/test/projects/django-hack/django/django/contrib/auth/management/__init__.py",
line 79, in create_permissions
Permission.objects.using(using).bulk_create(perms)
File "/Users/test/projects/django-hack/django/django/db/models/query.py",
line 438, in bulk_create
ids = self._batched_insert(objs_without_pk, fields, batch_size)
File "/Users/test/projects/django-hack/django/django/db/models/query.py",
line 1079, in _batched_insert
self._insert(item, fields=fields, using=self.db)
File "/Users/test/projects/django-hack/django/django/db/models/query.py",
line 1056, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File
"/Users/test/projects/django-hack/django/django/db/models/sql/compiler.py",
line 1094, in execute_sql
cursor.execute(sql, params)
File
"/Users/test/projects/django-hack/django/django/db/backends/utils.py", line
63, in execute
return self.cursor.execute(sql, params)
File "/Users/test/projects/django-hack/django/django/db/utils.py", line
93, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Users/test/projects/django-hack/django/django/utils/six.py", line
683, in reraise
raise value.with_traceback(tb)
File
"/Users/test/projects/django-hack/django/django/db/backends/utils.py", line
63, in execute
return self.cursor.execute(sql, params)
File
"/Users/test/projects/django-hack/django/django/db/backends/mysql/base.py",
line 88, in execute
return self.cursor.execute(query, args)
File
"/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/cursors.py",
line 221, in execute
self.errorhandler(self, exc, value)
File
"/Users/test/.virtualenvs/djangodev/lib/python3.6/site-packages/MySQLdb/connections.py",
line 38, in defaulterrorhandler
raise errorclass(errorvalue)
django.db.utils.OperationalError: (1366, '')
after much poking around, I believe the problem is the escaping in the
first three VALUES of the following statement:
INSERT INTO `auth_permission` (`name`, `content_type_id`, `codename`)
VALUES ('Can add \xc3\x83\xc2\x85ngstr\xc3\x83\xc2\xb6m\\'s Articles', 941,
'add_article'), ('Can change \xc3\x83\xc2\x85ngstr\xc3\x83\xc2\xb6m\\'s
Articles', 941, 'change_article'), ('Can delete
Re: Renaming the postgresql_psycopg2 backend
> On 24 Jan 2017, at 19:25, Tim Graham wrote: > > Okay, I updated the PR to use a deprecation. I'd rather not complicate things > with an accelerated deprecation. +1 I think that’s the right way. -- Aymeric. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9A05AB0F-B561-4F53-8D9C-BAE97032BCB6%40polytechnique.org. For more options, visit https://groups.google.com/d/optout.
Re: [ANNOUNCE] Django 1.11 alpha 1 released
Hey, I have recently started trying to contribute to Django. Wanted to know how can one help in testing for the final release? Using the new features in small projects or is there a formal procedure for testing? On Wednesday, January 18, 2017 at 6:48:18 AM UTC+5:30, Tim Graham wrote: > > We've made the first release on the way to Django's next major release, > Django 1.11! With two and a half months until the scheduled final release, > we'll need timely testing from the community to ensure an on-time and > stable release. Check out the blog post: > > https://www.djangoproject.com/weblog/2017/jan/17/django-111-alpha-1/ > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5fc4adab-2380-49c7-9839-5b371f9e516d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
