Re: standardised approach for packaged apps to define default settings

[Raphaël Barrois]

Hello,

On this topic, I'm using the excellent django-appconf library 
(https://pypi.python.org/pypi/django-appconf), which:
- Supports namespaced custom settings
- Handles default values natively

As far as I can tell, that solution is pretty efficient without requiring 
specific Django support ;)

On Wed, 1 Feb 2017 01:29:12 -0800 (PST)
Anthony King  wrote:

> Yesterday, I posted a new ticket asking about the potential for adding a 
> standardised approach for packages to define default settings. [0]
> 
> I was writing an app, and when looking for a way to set settings, there was 
> no clear solution.
> 
> 
> Looking at what other projects do, the goto was something like this:
> #myapp/settings.py
> from django.conf import settings
> MYAPP_SOME_SETTING = getattr(settings, 'MYAPP_SOME_SETTING', 'default')
> 
> This would then be accessed via myapp.settings, which will not reflect 
> dynamic updates (specifically, override_settings).
> 
> 
> What I feel is needed is this:
>   - have apps access their settings via django.conf.settings, always
>   - Allow to define default settings as cleanly as it is for settings to be 
> written for projects.
>   - Allow for dynamic settings that are calculated by default, but can be 
> overridden by the project. [1]
>   - *Nice to have: *This should ideally be written as a DEP so that it's 
> usable on Django 1.11
> 
> 
> *These are by no means final solutions. *I'm just looking to get the ball 
> rolling.
> 
> I'd like to propose a couple of solutions for this to discuss.
> 
> - First, Load default_settings.py during the setup sequence.
> I'm not sure how feasible this would be. What order would the settings app 
> loads? what happens if your setting depends on another setting?
> 
> - Second, This could be done in AppConfig.
> This introduces a fundamental new way of doing settings, and looks magical 
> when using it.
> 
> class MyAppConfig(AppConfig):
> ...
> class DefaultSettings:
> MYAPP_SOME_SETTINGS = 'default'
> 
> @property
> def MYAPP_DYNAMIC_SETTING(self):
> # set a default dynamically, based on environment or settings?
> 
> accessing settings.MYAPP_SOME_SETTINGS feels kind of strange though, as 
> there's no direct relation between this new class and the 
> django.conf.settings.
> 
> [0] https://code.djangoproject.com/ticket/27798
> [1] 
> https://github.com/YPlan/nexus/blob/620efbb7d13100848346c45717ca63d01ec301e5/nexus/conf.py
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20170201132910.5c09827d%40ithor.polyconseil.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Presenting DCP, a compatibility layer for Django (feedback welcome)

[Raphaël Barrois]

Hi Pascal,

I'm unsure as to what problem you're trying to solve; if I understand 
correctly, this project aims at making a "new"
Django behave like an "old" one.


I see a few possible use cases in the conversation:

* Running a legacy application (without new development) beyond the upstream 
support schedule (namely Django)
* Developing libraries (or reusable apps) that need to support several Django 
versions at once
* Using libraries that don't support (yet/anymore) your chosen Django version

In my opinion, the Django project has no reason to try to support the first 
one: those projects are basically
requesting longer security support from both Django and a set of third-party 
libraries; this requires time from the
maintainers of those projects.
If the project has decided to *not* provided longer-term-support, it is often 
for good reason, mostly "nobody is paying
developers enough for that additional support to be viable — if developers are 
paid at all".


For the second part, as a library developer, I'm indeed interested in a 
compatibility layer that would allow me to use
the *new* Django APIs in older versions; not the other way around.
This way, I can benefit immediately from improvements in the Django codebase 
and prepare my code for future upgrades —
akin to Python's ``from __future__ import ...```.
However, such a feature would need to provide a *wrapper* around Django's APIs 
and not modify it: other code in
the project is likely to rely on the documented behaviour.


For the third part, I'm unsure how you'd handle various libraries having 
different Django target versions?





On Mon, 23 Jan 2017 13:09:35 +0100
Pascal Chambon  wrote:

> Hi Dmitry,
> 
> thanks for the alternative way, however it seems more a *complement *than a
> replacement of DCP.
> 
> Indeed, your "rewriting" approach updates a codebase to support the latest
> django API, but it raises a number of issues :
> 
> - how could it be applied to django reusable apps, installed via pip ? Does
> it mean a package hook should be used to automatically transform code on
> "pip install" for django-related modules, as setuptools allows it for the
> 2to3 case ? Or a special import hook for django ?
> - it doesn't ensure backwards compatibility of the updated module, so
> unless every django-app maintainers use that system to stick to the latest
> django version, conflicts would remain when handling big projects.
> - imo it can only handle trivial changes (renames mostly) ; when features
> disappear, or are replaced by much different approaches, unless this
> rewritter itself injects big shims, it wouldn't work. For example, the
> SortedDict, in which keys could be inserted at arbitrary positions, has
> been replaced by stdlib OrderedDict which is "readonly", so the code around
> needs to be rethought accordingly.
> 
> That being said, I guess everyone would love it, if they could upgrade
> THEIR codebase semi-automatically, instead of doing mass regex
> search/replaces.
> There are plenty of AST modifiers for python (
> https://github.com/berkerpeksag/astor, or others, pytest does some nifty
> ast hacking too...), else a regex based django-command would already be
> nice too.
> 
> regards,
> Pascal Chambon
> 
> 
> 
> 
> 
> 2017-01-22 22:38 GMT+01:00 Dmitry Gladkov :
> 
> > Hi,
> >
> > Making Django upgrades less painful is a great goal, but I believe the
> > patching Django and restoring removed functionality is not the right
> > solution. JavaScript world has the same problem with far more frequent
> > major compatibility breakages and they solve it with automatic codebase
> > upgrade tools like jscodeshift [1]. This approach uses AST transformation
> > similar to what 2to3 does, but with configurable transformation rules.
> > There's also another project written in Python [2] that implements simpler
> > and more general, but less reliable regex-based approach.
> >
> > I believe adding codebase upgrade rules for each major Django release and
> > giving users ability to apply them by running something like  ./manage.py
> > upgrade_from 1.7 after Django upgrade will greatly simplify upgrading of
> > large Django projects.
> >
> > [1] https://github.com/facebook/jscodeshift
> > [2] https://github.com/facebook/codemod
> >
> > --
> > Best wishes,
> > Dmitry Gladkov
> >
> > On 22 January 2017 at 20:53, Pascal Chambon 
> > wrote:
> >  
> >> Hi,
> >>
> >> @James Pc - thanks for the support, if you happen to miss some fixers in
> >> DCP and don't have the opportunity to contribute them, please open an issue
> >> so that I have a look
> >>
> >> @Tim Graham & James James Bennett - from what I sum up, the new policy
> >> simply extends the delay between breaking changes, and so the overall life
> >> expectancy of django reusable apps, but other limitations remain as is.
> >> I've detailed the misc benefits that a compatibility layer would bring, I
> >> just hope not too many people needing big compatibility will fail to learn
> >> about djang

Re: Guidelines to name python modules of Django applications?

[Raphaël Barrois]

If I understand correctly the input, would the following proposal be an 
acceptable middle ground?

Replace:
> It’s often useful to prepend django- to your module name when creating a 
> package to distribute. This helps others
> looking for Django apps identify your app as Django specific

With:
> It's often useful to use the "Framework :: Django" setup.py classifiers when 
> creating a package to distribute.
> This helps others looking for Django apps identify your app as Django 
> specific.
>
> On a related note, using a different name between your PyPI package and the 
> actual module to import increases the risk
> of namespace collisions: if your package is called "django_foo" but imported 
> as "foo", it might conflict with a
> package named "foo". If possible, use the same name for the "module to 
> import" and the "packaged application".


I also think that the current tutorial is slightly confusing for newcomers: it 
suggests to use different names for the
package and the module — a distinction which might not be quite 
beginner-friendly.


-- 
Raphaël


On Thu, 08 Dec 2016 16:29:42 +
Michael Manfre  wrote:

> As some one who maintains django packages, I wouldn't use "django_" and
> don't think it should be an official recommendation. I do support
> documenting the potential collisions to let package maintainers make a more
> informed decision.
> 
> On Thu, Dec 8, 2016 at 7:17 AM Adam Johnson  wrote:
> 
> > +1 (to what Aymeric and Florian said)
> >
> > On 7 December 2016 at 19:54, Robert Roskam  wrote:
> >
> > +1
> >
> > On Wednesday, December 7, 2016 at 11:25:05 AM UTC-5, Marc Tamlyn wrote:
> >
> > What Aymeric and Florian sayid.
> >
> > On 7 December 2016 at 15:58, Florian Apolloner  wrote:
> >
> > On Wednesday, December 7, 2016 at 1:10:48 PM UTC+1, Aymeric Augustin wrote:
> >
> > I still think I wouldn't use a django_ prefix when I create new apps
> > because
> > it adds a small but pervasive overhead to prevent conflicts that aren't
> > common
> > in practice.
> >
> >
> > Same here, I am certainly against making that a recommendation from/in
> > Django itself.
> >
> > Cheers,
> > Florian
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-develop...@googlegroups.com.
> > To post to this group, send email to django-d...@googlegroups.com.
> > Visit this group at https://groups.google.com/group/django-developers.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/django-developers/ba0114d9-0baa-4662-8fff-f7c9b03b90d3%40googlegroups.com
> > 
> > .
> >
> > For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-developers+unsubscr...@googlegroups.com.
> > To post to this group, send email to django-developers@googlegroups.com.
> > Visit this group at https://groups.google.com/group/django-developers.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/django-developers/b6258a98-2f7e-4617-bb44-e7bc14952327%40googlegroups.com
> > 
> > .
> >
> > For more options, visit https://groups.google.com/d/optout.
> >
> >
> >
> >
> > --
> > Adam
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-developers+unsubscr...@googlegroups.com.
> > To post to this group, send email to django-developers@googlegroups.com.
> > Visit this group at https://groups.google.com/group/django-developers.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/django-developers/CAMyDDM1jBb50Rw74sEP8Ot7ykTDop7pn8Xr6kZw4uqgJmzot3Q%40mail.gmail.com
> > 
> > .
> > For more options, visit https://groups.google.com/d/optout.
> >  
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegr

Re: automating Django releases

[Raphaël Barrois]

On Fri, 25 Nov 2016 11:49:54 -0800 (PST)
Tim Graham  wrote:

> After doing releases about once a month for a while, I'm thinking it would 
> be nice if releasing Django could be a bit more automated. As far as I 
> know, the process hasn't changed too much in 10 years, while the state of 
> Python packaging has improved.
> 
> Currently doing a release requires bumping the VERSION tuple in 
> django/__init__.py [0], creating tarball/wheel and checksum files, asking 
> someone in IRC to verify the checks look fine, and upload files to various 
> locations [1]. My idea would be to use setuptools_scm [2] (that choice 
> because this is the only tool I know about) to eliminate the need to 
> bump/set a VERSION tuple and instead fetch the version information based on 
> git tags. Something like (based on what I found in djanog-hosts [3]):
> 
> __version__ = pkg_resources.get_distribution('django').version
> 
> One issue I'm not sure how to deal with is how to provides 
> backwards-compatibility for django.VERSION, the tuple format that looks 
> something like:
> 
> VERSION = (1, 11, 0, 'alpha', 0)
> 
> I think removing or deprecating it would be quite disruptive for the 
> community as I've seen lots of third-party apps that do checks like "if 
> django.VERSION < (1, 9): ...", for example. I suspect it's possible to 
> parse the version string to create something similar, but I wondered if 
> anyone has experience with that.
> 
> Finally, uploading to PyPI and djangoproject.com (release and checksum 
> files) could be done automatically by Jenkins or something similar.
> 
> Thanks for your comments, concerns, and suggestions.
> 
> [0] 
> https://github.com/django/django/blob/6252fd6314f66d2d303cc47c791ffefd27169b42/django/__init__.py#L5
> [1] 
> https://docs.djangoproject.com/en/dev/internals/howto-release-django/#actually-rolling-the-release
> [2] https://github.com/pypa/setuptools_scm
> [3] 
> https://github.com/jazzband/django-hosts/blob/0eae77b31927ddb0267943d6f9bcfc22f14706f2/django_hosts/__init__.py#L11
> 

Hi,

The process I use goes the other way around: with an automated tool, we type 
``make release-minor`` and get:
1. Update the VERSION tuple
2. Update the ChangeLog with a release date
3. Create the adequate git tag
4. Prepare the ChangeLog for the next release

The tool we use is not open-source yet, however it seems very similar in design 
to https://pypi.python.org/pypi/zest.releaser.

Moreover, this pattern does not change the way your package is installed or 
detects its version number: that number is always available in the source code,
no matter how the package was installed.

-- 
Raphaël

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20161125221633.3a5d1166%40anoth.
For more options, visit https://groups.google.com/d/optout.

Re: #27485 new New feature Allow ALLOWED_HOSTS to accept an IP-range / wildcard

[Raphaël Barrois]

On Wed, 16 Nov 2016 20:32:34 -0800 (PST)
Yo-Yo Ma  wrote:

> I'm not a fan of adding more complexity, for a couple reasons:
> 
> 1) you have the ['*'] option for local / office development
> 2) you can just add a record to /etc/hosts to point to the server and then 
> use a name like e.g., local-office - just
> update the record to point to whatever IP you're using st the time
> 

Since settings are Python code, you could also do the following:

ALLOWED_HOSTS = ['.mydomain.com'] + ['192.168.1.%d' % i for i in range(256)]

With this flexibility in mind, I don't think increasing the amount of wildcard 
options is required ;)

-- 
Raphaël

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20161117093831.28cc1cd2%40ithor.polyconseil.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Making Django more PaaS-friendly

[Raphaël Barrois]

Hi James,

>From the experience on our projects, the ``CONFIG.getstr('db.password')`` 
>format works well in settings files:
- It makes it clear that this field is loaded from the environment, and what 
type is expected
- The setting is set and loaded in the same place
- It allows for type checking
- It handles loading other values

This is easier than automagically altering settings based on environment 
variables, as the user might set a value in
some part of its settings file, yet find it overridden by some later 
declaration of ``ENV_SETTINGS``.



In my opinion, a few important questions to address would be:
- What is the format of those environment variables? For instance, using a 
'DJANGO_' prefix will avoid name conflicts
- Can developers use this "environment configuration parser" tool to parse 
extra, custom settings; or are we limited to
  Django's needs?
- Should we only support "loading from the environment", or is it useful to 
allow loading from a ini-style file
  (typically to override some values on a local development environment)?


We wrote a small helper to load configuration from various sources (environment 
and configuration files); this could be
helpful for designing this new feature: 
http://getconf.readthedocs.org/en/latest/

Feel free to ping me if you're interested in more feedback from building and 
using that kind of setup :)


On Sun, 10 Apr 2016 23:33:46 -0700
James Bennett  wrote:

> Apologies for how late in the process this is; job hunt + moving
> cross-country ate a lot of my time.
> 
> At Django Under the Hood I proposed that for Django 1.10 we try to find
> some common best practices for deploying Django on popular
> platform-as-a-service (PaaS) solutions, and building support for them into
> Django. The biggest one of these is probably having the ability to read
> configuration from environment variables instead of hard-coding things into
> settings files.
> 
> At the very least I'd like to propose (assuming Kenneth is on board with
> it) integrating dj-database-url[1] or something like it directly into
> Django, so that there's no longer a third-party dependency for reading the
> database configuration from an environment variable. Whether this is just
> porting dj-database-url itself in, or making the DATABASES setting
> understand URLs, I'm unsure of yet and would be interested to hear feedback
> on. Either way I'm willing to put in the time to develop the patch.
> 
> More generally, I'd like to see Django grow helpers for specifying settings
> that should be read from environment variables, and which environment
> variables to use (the email settings are another common case, as is
> anything that requires an API key to access).
> 
> There are a few ways to design this. One option would be just a minimal
> wrapper around os.getenv, perhaps taking an optional type or type-coercing
> function, so that it would be possible in a settings file to do:
> 
> SECRET_NUMBER_SETTING = env_setting('SECRET_NUMBER', int)
> 
> However, this is not much better than the current practice of calling
> os.getenv. A better solution might be the ability to specify a group of
> settings which will be read from the environment, and have Django
> automatically read and set them. For example:
> 
> ENV_SETTINGS = [
> ('SECRET_NUMBER_SETTING', int),
> ('ACME_API_KEY', str),
> ('VOLCANO_LAIR_PASSWORD', str),
> ]
> 
> would read the named settings from those environment variables, and coerce
> them to the appropriate types using the function provided.
> 
> The main problem with this is that it's really not very elegant. But at the
> moment I can't think of anything better, and so I'd like to throw the floor
> open to ideas on nicer approaches to this. If one can't be found, I do
> think Django 1.10 should at least figure out how to handle database config
> from env since that's such a common use case nowadays, but ideally we'd be
> able to pin down a good API for generically pulling configuration from the
> environment.
> 
> 
> [1] https://github.com/kennethreitz/dj-database-url
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160411143238.6565a96a%40ithor.polyconseil.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Bringing some popular must have django tools/packages under django org umbrella

[Raphaël Barrois]

On Wed, 16 Mar 2016 15:39:40 +0100
James Pic  wrote:

> On Wed, Mar 16, 2016 at 3:22 PM, Matías Iturburu  wrote:
> > Even if I submit a patch I wouldn't be able to:
> > - Merge it into mainline.
> > - Upload the patched version to pypi.
> >
> > So it's a no-starter,
> > I can't rely on my fork, not for production, as I
> > should guarantee that the package it's, at least, as tested as mainline,
> > which usually involves non-trivial infrastructure.  
> 
> If I understand correctly, you need a patch, but you don't want to do
> it, because you can't have it released right away.
> 
> Your work is to have a feature in a project, you want to use an app,
> but you don't want to contribute to it. If I understand correctly, you
> want volunteers to do the work for you so that you get your project
> done and perhaps even get paid, and not give anything in return.
> 
> Good patches are quickly merged. If it has tests, and good code
> coverage, and supports new versions of Python, I can tell you it's not
> going to take long before your patch is merged in most cases.
> 
> Otherwise, yeah, just publish your fork on PyPi until it's merged
> upstream. I don't see what's the problem here. We've had the case in
> django-cities-light were a user implemented Region support, published
> it in django-cities-reducedfat or something (lol), and then
> contributed upstream, and then we released the contribution in
> django-cities-light. What's the issue here ?
> 
> >>
> >> I know the first portings are hard but once you've ported a dozen it
> >> becomes piece of cake so don't be afraid of trying ! And please contribute
> >> to the apps !!
> >>
> >> I know some people who only open issues and never submit a patch on
> >> github, isn't that super annoying?  
> >
> > Really? you go around asking users to do the work of maintainers?  
> 
> I don't understand, what do you mean work ? Are you paying the
> maintainers to maintain their projects ? Are you talking about
> dual-licensed projects like django-suit where you payed an Enterprise
> license for ?
> 
> Well what do you prefer, that maintainers shut down a project because
> they don't have time or motivation to keep on, or do you prefer that
> the project lives on with community support ?
> 
> If I understand your logic correctly, you should re-implement the
> features you were using from an app in your own project to drop the
> dependency on the app that you consider un-maintained. Either way, you
> end up doing your job, but if you contribute then it's for everybody,
> not just your project.
> 
> Seems like "Open Source" doesn't wrok the same in your world and in mine :)
> 
> Perhaps if you were maintaining Open Source apps you'd understand. I
> don't know if you use any things like torrents, but do you know what
> the "leecher" concept is like ? It's when a user downloads and then
> doesn't share.
> 
> I don't think I have anything to add here, I'm sure more experienced
> hackers will find better phrasings than me for this. I may sound
> harsh, but really I'm not, I'm just trying to understand how your
> logic works and so far it seems broken for me so I'd really like to
> understand.
> 
> Also, about django-endless-pagination, I tried it once, but then
> decided to go on with my own 5 sloc of JS implemantion of endless
> pagination, using Django's normal pagination on the python side, so
> I'd like to be convinced that it's really necessary in a project ! But
> that I guess is another topic.
> 
> Best ;)
> 
> James
> 


Hello there,

I am not sure the tone of that discussion is suitable for that mailing-list ;)


Regarding the initial discussion, if I understand correctly, the idea would be 
to find a way to prevent useful packages
from turning into abandoned, unmaintained projects.

>From my (personal) experience:
- If someone is interested in taking over maintenance of an abandoned project, 
the original author is often quite happy
  to help
- If the task seems too daunting to handle it single-handedly, you could try to 
find a few other power-users of the
  package who would be willing to help maintain it

In that second case, the benefit of a generic/umbrella organisation would be to 
simplify the task of finding
co-maintainers for a project.

I am not part of the Django team, but I don't see why the maintenance of those 
extra projects would have to be handled
by the Django team; a better start might be to approach the maintainers of a 
few projects that matter to you and
discuss the idea with them; if that works, you could, as a group, decide 
whether you want that to happen within the
Django org or within a different org.


-- 
Raphaël

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups

Re: remove support for unsalted password hashers?

[Raphaël Barrois]

Hi Tim,

A few notes here:
Just as djangoproject.com might need to keep those old hashers around, many 
projects will need it as well.
As such, providing the hashers in a dedicated "legacy" package might be the 
solution for most projects.

For the deprecation process, I think the needs of most sites would be:
1) Find out how many accounts use deprecated hashes, and when they last logged 
in
2) Based on that information, decide which hashers can be removed, and which 
accounts need to have their password
reset.
Do you think this should be provided as a management command (useful as Django 
improves its hashers over the years),
or simply as a few code snippets in the release notes?

Finally, I suggest that the "no-op test hasher" retains some properties of the 
usual hashers, mainly "password is
transformed" and "any length is accepted".
Indeed, I have seen many issues with developers using ``user.password = 'foo'`` 
instead of going through
``user.set_password``; which is quickly discovered when going through the usual 
test setup.
Also, some users test for arbitrarily long passwords, which are perfectly fine 
with normal hashers and shouldn't thus
fail in a test setup due to a "no-op cleartext hasher".


If you're interested, I can provide some help with the deprecation 
documentation and no-op code in the next few days,
depending of which options you choose to go with.


-- 
Raphaël


On Wed, 3 Feb 2016 12:26:00 -0800 (PST)
Tim Graham  wrote:

> Acknowledged Donald, I just didn't want to bite off too much at once.
> 
> I think the unsalted hashers removal could be done as a 
> backwards-incompatible change. I wrote up some documentation including 
> queries to check if your database is affected: 
> https://github.com/django/django/pull/6082
> I'll be curious to know if anyone has a project that started in the Django 
> 0.90 era which returns some results for those queries.
> 
> About removing the SHA1PasswordHasher, MD5PasswordHasher, and/or 
> CryptPasswordHasher -- I suspect many more users will be affected, so the 
> normal deprecation process seems appropriate. To give an example, 8,484 
> (64%) of the passwords for djangoproject.com users are SHA1. If the SHA1 
> hasher is deprecated, what would we do? Options I can think of:
> 
> 1. copy the hasher into the djangoproject.com source
> 2. release the legacy hashers as a separate package for those projects that 
> need them
> 3. mark old passwords as unusable and force a reset if one of those users 
> comes back
> 
> The max "last login" for a user with a SHA1 hash is February 2013.
> 
> Also, the MD5PasswordHasher is suggested in the documentation as a way to 
> speed up tests so we would need to change that, whether it's force_login() 
> or some new "no-op test hasher" .
> 
> On Tuesday, February 2, 2016 at 2:20:44 PM UTC-5, Donald Stufft wrote:
> >
> >
> > On Feb 2, 2016, at 1:52 PM, Tim Graham > 
> > wrote:
> >
> > Just to be clear, my proposal here is only about removing 
> > UnsaltedSHA1PasswordHasher and UnsaltedMD5PasswordHasher. The salted 
> > versions of these hashers remain.
> >
> >
> >
> > It seems silly to remove the unsalted options and leave the salted 
> > options, they are basically equally [1] as secure since computational power 
> > is such that it is, that it’s not really worth it to use rainbow tables 
> > anymore anyways.
> >
> > [1] Ok, Ok, technically salted are a wee bit more secure, but given that 
> > you can compute the MD5 of every single possible lower case alpha numeric 
> > of 6 characters or less in under a minute on a single regular 
> > desktop/server.. I don’t believe the distinction is useful.
> >
> > -
> > Donald Stufft
> > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 
> > DCFA 
> >
> >  
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160204182155.7c3bba5e%40ithor.polyconseil.fr.
For more options, visit https://groups.google.com/d/optout.

Re: remove support for unsalted password hashers?

[Raphaël Barrois]

Hi Tim,

I would suggest removing those hashers from the default list, but keeping at 
least the
django.contrib.auth.hashers.UnsaltedMD5PasswordHasher around.

That hasher, being the fastest non-plaintext hasher around, is quite useful 
when running tests: it allows login checks
to be performed much faster.

Beyond this, the idea seems great — it's still pretty easy for a site to keep 
them around if it needs them.


-- 
Raphaël

On Tue, 2 Feb 2016 08:10:50 -0800 (PST)
Tim Graham  wrote:

> Django 0.90 stored passwords as unsalted MD5. Django 0.91 added support for 
> salted SHA1 with automatic upgrade of passwords [0].
> 
> In Django 1.4, the new password hashing machinery was added and some users 
> complained that they couldn't upgrade because the password format from 
> Django 0.90 was no longer accepted (passwords encodings starting with 
> "md5$$" or "sha1$$", though the ticket suggests Django never used the 
> latter prefix) [1].
> 
> I wonder if it's about time to remove these hashers [2]? I think it'd be 
> okay for users who haven't logged in since Django 0.90 to reset their 
> password (assuming the site provides that mechanism). I would consider 
> recommending that site administrators mark any unsalted passwords 
> "unusable" to mitigate the possibility of leaking unsalted passwords in the 
> event the database is compromised.
> 
> I think this is as simple as:
> 
> users = User.objects.filter(password__startswith='md5$$')
> for user in users:
>  user.set_unusable_password()
>  user.save(update_fields=['password']
> 
> [0] https://code.djangoproject.com/ticket/18144#comment:18
> [1] https://code.djangoproject.com/ticket/18144
> [2] 
> https://github.com/django/django/compare/master...timgraham:remove-unsalted-hashers
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160202182336.719be5ca%40ithor.polyconseil.fr.
For more options, visit https://groups.google.com/d/optout.

Re: re-thinking middleware

[Raphaël Barrois]

On Fri, 8 Jan 2016 11:38:04 -0700
Carl Meyer  wrote:

> Thanks everyone for the reviews. I've pushed a new commit addressing
> most of the issues raised:
> https://github.com/django/deps/commit/62b0ee351727cb0e7ef41ba6dd2f3f7280a219de
> 
> More comments and replies (to various people in the thread) below:
> 
> On 01/08/2016 09:01 AM, Florian Apolloner wrote:
> > PR1 (allows changing the middlewares during the middleware run): 
> > https://github.com/django/django/pull/5591
> > PR2 (static middleware chain, seems simpler and probably the way to
> > go): https://github.com/django/django/pull/5949/
> 
> Yes, the latter is what Pyramid does and what I had in mind; in fact,
> its performance benefits are one of the justifications given in the
> DEP for why to use the factory approach rather than simple functions.
> I don't think "changing middlewares per request" is a use case that's
> worth the performance implications (and it's not a use case we
> currently support anyway).
> 
> Thanks for the super-quick turnaround on implementation!
> 
> On 01/08/2016 10:35 AM, Aymeric Augustin wrote:
> > The only (and minor) concern I have is about allowing function-based
> >  middleware factories to return None.
> > 
> > In the spirit of “there should be only one way to do it”, I would 
> > require raising MiddlewareNotUsed explicitly to disable a
> > middleware. A middleware factory that returns None would cause an 
> > ImproperlyConfigured exception. Otherwise middleware could be
> > skipped by mistake, if the developer forgets to return the
> > middleware from the factory. This is especially a concern for
> > production-only middleware that developers can’t run locally.
> [snip]
> 
> Yes, you and Ryan are absolutely right, allowing `None` was a mistake
> and I've removed it from the spec. For function-based middleware,
> there's also the option to simply return the ``get_response`` callable
> you were given, which has the same effect of "skipping yourself." That
> falls out naturally from the concept, so I don't see any reason to
> disallow it, but we don't need to mention it in the docs if we want to
> stick to "one way to do it."
> 
> On 01/08/2016 04:51 AM, Markus Holtermann wrote:
> > I would, however, include the exception handling in the examples 
> > provided in section "Specification" as that is an integral part of 
> > middlewares, too.
> 
> Good idea, I've made that change in the commit linked above.
> 
> > Nitpicking, I would also name the settings variable MIDDLEWARES 
> > (i.e.plural) as it is a list of middlewares, not just one.
> 
> Well, this is a matter of some debate :-) See e.g.
> https://github.com/rack/rack/issues/332, where a number of people
> fervently argue that "middleware" is a "mass noun" along the lines of
> e.g. "furniture", and that it is even incorrect to say "a middleware"
> (much like you would never say "a furniture"); instead we should
> always say "a middleware component."
> 
> I think those people are fighting a lost battle; the usage of
> "middleware" as singular is already well established in the Python
> world, even outside Django; people frequently talk about "a WSGI
> middleware."
> 
> That said, my ear still prefers "middleware" as both the singular and
> the plural (there are such words in English) and dislikes
> "middlewares." So I'd prefer to stick with MIDDLEWARE and haven't
> changed it in the spec. But if most people think MIDDLEWARES is
> better, I won't stand in the way.
> 
> We could also go with something like MIDDLEWARE_FACTORIES or
> MIDDLEWARE_PIPELINE and avoid the issue altogether :-)
> 
> Carl
> 



Hi,

I've got only one minor suggestion to the "backwards compatibility"
section of the DEP.

> It currently states that "If the ``MIDDLEWARE`` setting is provided
> [...], the old ``MIDDLEWARE_CLASSES`` setting will be ignored.

I suggest that, instead, this fails loudly if both ``MIDDLEWARE`` and
``MIDDLEWARE_CLASSES`` are set.
This would prevent projects from keeping the two versions around and
having to remember which one is currently in use.


Beyond that, I love the overall design of this upgrade :)

-- 
Raphaël

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160111005458.796853f0%40corvis.
For more options, visit https://groups.google.com/d/optout.

Re: Extending JSONField serialization

[Raphaël Barrois]

On Wed, 6 Jan 2016 10:27:01 -0500
Michael Manfre  wrote:

> On Wed, Jan 6, 2016 at 9:58 AM, Tom Christie  wrote:
> 
> > Customizing the encoder (or even using DjangoJSONEncoder by default) isn't
> > so bad.
> >
> > I'm less convinced about the usefulness of customizing the decoder - once
> > you've encoded the data into JSON any additional type information is lost,
> > so casting back to python primitives is always going to need to be handled
> > explicitly.
> >
> 
> Whether or not a configurable decoder seems useful for the situations we
> can think of, only allowing users to configure half of the encode/decode
> cycle seems odd to me. Allowing both to be configurable requires a trivial
> amount of extra effort to implement and maintain.
> 
> Regards,
> Michael Manfre
> 

Indeed, this is only helpful if the user is able to customize both encoding and 
decoding of its messages.
For instance, on one of our projects, we use a custom serializer that encodes 
complex types as {"__type__": "foo", "value": "bar"} dicts.
Our deserializer can then handle the decoding of such fields to a native Python 
datastructure.

This is possible with "jsonfield" (https://pypi.python.org/pypi/jsonfield) 
which allows to set a custom encoder/decoder.

Coming back to Aymeric's question, I'd love to be able to set a custom 
decoder/encoder for a JSONField.
Being able to only specify the encoding is error-prone, since my code will then 
have to handle the distinction
between "this is a value set pre-save, so it's in a native type" and "this 
time, it's from the database, so I have to cast it back".


Regards,

-- 
Raphaël Barrois

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160106173748.43400622%40ithor.
For more options, visit https://groups.google.com/d/optout.

Re: Provide a way to pass kwargs when initializing the storage class

[Raphaël Barrois]

Hello,


The core of the proposed solution seems quite interesting; however, it also 
introduces a new configuration format for backends.

Caches and databases use a dict with a "BACKEND" key and an "OPTIONS" dict for 
kwargs to pass to the backend.
Likewise, entries in the TEMPLATES list are dicts with a "BACKEND" key and 
various options.


Do you think that the new setting should match these options instead of the 
proposed two-tuples?


-- 
Raphaël


On Sat, 7 Nov 2015 12:15:49 +0100
Aymeric Augustin  wrote:

> Hello,
> 
> Currently the DEFAULT_FILE_STORAGE and STATICFILES_STORAGE settings
> contain a dotted Python path to the storage class. The class is
> instantiated without any arguments.
> 
> 
> ** Problem **
> 
> This leads to three annoyances.
> 
> 1) Third-party libraries like django-storages(-redux) need to provide
> a setting for every value that could be required to configure the
> storage. This additional level of indirection is annoying.
> 
> If you’re skeptical, look at
> https://github.com/jschneier/django-storages/blob/f2880b16b57a36b241a54932255e1a852c0e0ac7/storages/backends/s3boto.py#L204
> .
> 
> 2) Compounding this problem, third-party libraries that want to use a
> Django storage backend, but likely not the default one, have no
> convenient way for the user to configure the storage.
> 
> Having settings such as `MY_THIRD_PARTY_APP__AWS_ACCESS_KEY_ID` etc.
> for every possible storage-related setting sounds unreasonable.
> 
> (This is the problem I’m facing right now with one of my libraries.)
> 
> 3) The standard pattern for working around these problems is
> boilerplate-ish:
> 
> my_config = {foo: bar}
> 
> class ConfiguredStorageClass(GenericStorageClass):
> def __init__(self, *args, **kwargs):
> kwargs.update(my_config)
> super().__init__(*args, **kwargs)
> 
> DEFAULT_FILE_STORAGE = 'path.to.ConfiguredStorageClass'
> 
> 
> ** Proposed solution **
> 
> To solve this problem, I would like the DEFAULT_FILE_STORAGE and
> STATICFILES_STORAGE settings to accept an additional format: a 2-uple
> of (dotted Python path to class, init_kwargs).
> 
> This would allow simplifying the example above to:
> 
> DEFAULT_FILE_STORAGE = ‘path.to.GenericStorageClass’, {foo: bar}
> 
> 
> ** Variants **
> 
> We could go a bit further support 2-uples with args, 2-uples with
> kwargs, and 3-uples with args and kwargs. I didn’t propose it because
> I’m not sure this possibility adds much value. Arguments can be
> passed as kwargs even if the init method expects them positionally.
> 
> We could support setting DEFAULT_FILE_STORAGE and STATICFILES_STORAGE
> to an already initialized instance of a storage class. I’m against
> this idea because settings should remain primitive Python values
> whenever possible.
> 
> We could introduce DEFAULT_FILE_STORAGE_INIT_KWARGS and
> STATICFILES_STORAGE_INIT_KWARGS. Well. Perhaps not :-)
> 
> 
> What do you think?
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20151107125832.440abc17%40corvis.
For more options, visit https://groups.google.com/d/optout.

Re: Migrations and swappable models/AUTH_USER_MODEL

[Raphaël Barrois]

On Wed, 8 Jan 2014 20:27:54 +0200
Shai Berger  wrote:

> On Wednesday 08 January 2014 20:00:25 Andrew Godwin wrote:
> > > Instinctively, I'm almost -1 on 2); I'm not too happy about 1)
> > > either. If a model says it has a FK to auth.User, that shouldn't
> > > mean anything other than
> > > auth.User. I don't see that as cleanliness -- it's effectively
> > > monkeypatching.
> > > 
> > > 1b seems to me like the most reasonable option.
> > 
> > It's also the least user-friendly option. I don't see the same
> > problem with 2) - sure, the model says it has an FK to auth.user,
> > and your settings say that auth.user is swapped out in favour of
> > another model. I think it would actually be worse if we kept it
> > like it is - since auth.user is swapped out, it shouldn't have a
> > table made for it, so how are you even going to make an FK to it?
> > (this is how migrations currently fails - there's no auth.user left
> > for it to refer to).
> > 
> 
> I don't see how "you need to remember that auth.User really means
> some model defined in settings" is more user friendly than "you need
> to remember that __swappable__.X means that X is swappable". More so,
> as people already use auth.get_user_model() -- so, you can
> monkeypatch *that* when generating migrations, and let them just keep
> doing what they do today.

Actually, people don't use auth.get_user_model() in their models declaration, 
they use ``settings.AUTH_USER_MODEL``,
as described in the documentation: 
https://docs.djangoproject.com/en/1.6/topics/auth/customizing/#referencing-the-user-model

The actual user model may not be prepared at the time the ``models.py`` is 
loaded, thus failing to import.
>From a migrations point of view, this means that the ForeignKey is declared 
>exactly as if the user had actually, manually written 
>``ForeignKey('auth.User')`` instead of ``settings.AUTH_USER_MODEL``.

I think that's why Andrew suggests to either use a special placeholder 
(``__swapped__.X``) or to automatically detect swapped models from their actual 
``class Meta: swappable=True`` declaration.

> 
> > > However, I've had a different thought about this:
> > > > As long as you're not using any third-party apps, then
> > > > everything works fine
> > > 
> > > No, it doesn't really. You can't change the user model as part of
> > > the project
> > > evolution. Supporting this raises a whole set of additional
> > > problems -- even
> > > with 1b, not to mention the scenarios where we try to guess
> > > swappables from concrete models that happen to be their sources
> > > or targets.
> > 
> > Changing the user model during the project's lifespan is a
> > different task. Migrations will nominally support it - in that
> > it'll change all your FK constraints to immediately point to the
> > new table - but you still have to manage moving the data into that
> > table yourself, and I'm not sure we can do it any other way.
> > Migrations sees you changing the user model as an actionable
> > change, since as far as it's concerned you changed a load of "to"
> > arguments of foreignkeys when you changed the setting, and so it
> > can make migrations for that.
> > 
> 
> No, not with any of the three suggestions. The migrations machinery
> doesn't introspect the database to find the FK targets -- it re-rolls
> the migrations in memory; and if any of the suggestions is taken,
> then when you run the migrations with the setting pointing at some
> model, it will seem like it has always been that way.
> 
> > The problem is when the migrations are pre-made - i.e. from
> > third-party apps - and that's the issue I'm trying to solve.
> > 
> 
> The two problems are in conflict -- for evolving the user model,
> history must matter; for pre-made migrations, the choice of user
> model needs to look like it has always been the way it is "now" (at
> the time the migrations are run).

Actually, you have two problems with a third-party app:

- It needs to have a way to specify "I want a FK to whatever the user has 
chosen for a AUTH_USER_MODEL", no matter the *type* of its PK field.
=> This mustn't depend on what the third-party-app developer's actual 
AUTH_USER_MODEL setting was while generating the migration
- Ideally, if the user changes the model, this should be helped by migrations, 
for instance by put a warning e.g "Looks like you're changing a swappable 
model, where should I put automatic related migrations ? (these migrations 
shouldn't go into the app with the FKey to AUTH_USER_MODEL, which may be out of 
the user's control — e.g third party apps).

Another problem is apps that provide a swappable model whose default is 
AUTH_USER_MODEL but that may be customized through their own setting,
for instance ``return getattr(settings, 'MY_TARGET_MODEL', 
settings.AUTH_USER_MODEL)``.


-- 
Raphaël

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it,