Re: Referrer Policy Delivery, Django shouldn't do strict referrer check anymore?

[Flávio Junior]

Yes Jon, makes sense, sorry for missing that.
The only HTML-only solution I see for this is to manually 
add rel="noreferrer" to all external links on my webapp, which is a pain.
With extra backend code, one might also implement something similar 
to https://anon.click/ to prevent referrer leaking.

So the problem with Django strict referrer check is deeper. It introduces 
complexity to make secure webapps that don't leak referrer info.

PS: don't know if anon.click is safe or trustable.

Em sexta-feira, 4 de dezembro de 2015 16:52:26 UTC-3, Jon Dufresne escreveu:
>
> On Wed, Dec 2, 2015 at 10:29 AM, Flávio Junior  > wrote: 
> > Also, I can't imagine now why, but some 
> > developer might want to disable referer header altogether, and can 
> easily do 
> > so by setting policy to No Referrer. 
>
> Why is it unimaginable that I may want to maximize privacy for my 
> users? The domain my users are coming from is between me and my users. 
> If CSRF would work securely without any referer header (and not leak 
> the same data some other way), I would choose to disable it entirely. 
> Often the domain is enough to leak what kind of data a user was 
> reading or interacting with. For example, what might one assume 
> (rightly or wrongly) if the referer header was 
> "https://wikileaks.org";? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0a8cc4a7-76c6-434a-8ccb-f6d043638d34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Referrer Policy Delivery, Django shouldn't do strict referrer check anymore?

[Jon Dufresne]

On Wed, Dec 2, 2015 at 10:29 AM, Flávio Junior  wrote:
> Also, I can't imagine now why, but some
> developer might want to disable referer header altogether, and can easily do
> so by setting policy to No Referrer.

Why is it unimaginable that I may want to maximize privacy for my
users? The domain my users are coming from is between me and my users.
If CSRF would work securely without any referer header (and not leak
the same data some other way), I would choose to disable it entirely.
Often the domain is enough to leak what kind of data a user was
reading or interacting with. For example, what might one assume
(rightly or wrongly) if the referer header was
"https://wikileaks.org";?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADhq2b62tY-izLJ2WrgsskJmbBT5ArezGL17HuinbHv4hYAg3g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Referrer Policy Delivery, Django shouldn't do strict referrer check anymore?

[Flávio Junior]

Hi Collin,
Firefox doesn't include Origin header on same-origin POST/PUT/DELETE 
requests. I just tested it and this SO answer says the same 
.
But yes, checking both Origin and Referer headers would help giving 
support Origin When Cross-Origin.

I can create a ticket suggesting Django to check Origin header before 
checking Referer. Or do you want to create that Collin?

Em sexta-feira, 4 de dezembro de 2015 14:54:36 UTC-3, Collin Anderson 
escreveu:
>
> Also, if we checked the origin header, would it allow us to at least 
> support the "Origin When Cross-Origin" policy in all browsers? (Use the 
> Origin header for Safari and the referrer for all of the other browsers?)
>
>
> On Fri, Dec 4, 2015 at 10:38 AM, Tim Graham  > wrote:
>
>> Flávio, thanks -- since you seem to have a good understanding of the 
>> limitation, could you submit a documentation patch (or even just provide 
>> some draft text here)?
>>
>> On Friday, December 4, 2015 at 8:25:35 AM UTC-5, Flávio Junior wrote:
>>>
>>> Found a issue that already discusses this: 
>>> https://code.djangoproject.com/ticket/16870#comment:10
>>>
>>> Em quinta-feira, 3 de dezembro de 2015 13:41:09 UTC-3, Flávio Junior 
>>> escreveu:

 Florian, then Django will have to keep this limitation: can't use a 
 global no-referrer policy on HTTPS because of strict referrer check. 
 Correct? Should I create an issue to keep this logged?

 Em quinta-feira, 3 de dezembro de 2015 13:19:38 UTC-3, Florian 
 Apolloner escreveu:
>
>
>
> On Wednesday, December 2, 2015 at 7:37:30 PM UTC+1, Flávio Junior 
> wrote:
>>
>> If Django still needs 
>>  the strict 
>> referrer check, maybe a better error message should be implemented.
>>
>
> I do not see any reason why it would not need it anymore. Django needs 
> to ensure that the request came from a "trusted origin", therefore we 
> still 
> need it.
>
 -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/c0ce17a5-3f0e-4218-a2c8-cecf7da07fb9%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5f7b5a6e-0096-47ad-ad61-373a2d183c7c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Validation of m2m

[Federico Capoano]

It could be a potential ticket to work on my next django dev sprint.

But first it would be nice to have some basic consensus on how to proceed.

Was it ever discussed in any older thread or ticket?



On Thursday, December 3, 2015 at 5:21:06 PM UTC+1, Tim Graham wrote:
>
> Here's an open ticket about adding model level validation for 
> many-to-many: https://code.djangoproject.com/ticket/12938
>
> On Thursday, December 3, 2015 at 11:04:22 AM UTC-5, Federico Capoano wrote:
>>
>> Thanks Aymeric,
>>
>> I decided to post this problem on django-developers because I've read 
>> this ticket:
>> https://code.djangoproject.com/ticket/24731
>> Sorry for omitting this information.
>>
>> Has there been a discussion about this topic already?
>>
>> Would it be hard to implement an easier solution into django?
>>
>> I spent a few hours working on this issue, I consider myself fluent with 
>> django and it's quite shocking I had to put such an amount of effort just 
>> to validate many2many relationships before they are saved to the database.
>>
>> IMHO it would be better if we could do one of these two (or even both) 
>> things:
>>
>> 1. make this process easier in future django versions
>> 2. document the current best practice to solve this problem in current 
>> django to save people's time
>>
>> What do people you think?
>>
>> Here's my solution working with django 1.9:
>> https://github.com/openwisp/django-netjsonconfig/compare/4082988...master
>>
>> What do you think of it? Can it be improved in some way?
>>
>> Federico
>>
>>
>> On Thursday, December 3, 2015 at 1:43:21 PM UTC+1, Aymeric Augustin wrote:
>>>
>>> Hello Frederico,
>>>
>>> It appears that you're hitting the problem described in the "Avoid 
>>> catching exceptions inside atomic!" warning on this page:
>>>
>>> https://docs.djangoproject.com/en/1.8/topics/db/transactions/#handling-exceptions-within-postgresql-transactions
>>>
>>> To obtain that sort of result, I suppose you must be catching an 
>>> IntegrityError, re-raising a ValidationError, which Django in turn catches, 
>>> and then you hit that traceback.
>>>
>>> Adding an atomic block inside your try/catch block that catches the 
>>> IntegrityError will resolve that particular problem — putting that part of 
>>> the discussion into django-users territory.
>>>
>>> If this isn't what happens, please post your code and ask for help on 
>>> django-users.
>>>
>>> -- 
>>> Aymeric
>>>
>>> 2015-12-03 13:28 GMT+01:00 Federico Capoano :
>>>
 Hi everybody,

 I am sure it has happened to many of you.

 Validating m2m BEFORE saving the relationships is very hard and time 
 consuming.

 Now this solution:
 http://schinckel.net/2012/02/06/pre-validating-many-to-many-fields./

 Proposes to solve it with a ModelForm in the admin.
 Cool, that works.

 But, if I want to enforce validation on the model, to avoid corrupted 
 data, I notice the signal is executed in a transaction block, in which if 
 I 
 raise a ValidationError I get the following:

 Traceback (most recent call last):
   File 
 "/var/www/django-netjsonconfig/django_netjsonconfig/tests/test_device.py", 
 line 106, in test_m2m_validation
 d.templates.add(t)
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/fields/related_descriptors.py",
  
 line 843, in add
 self._add_items(self.source_field_name, self.target_field_name, 
 *objs)
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/sortedm2m/fields.py",
  
 line 138, in _add_items
 for val in vals:
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
  
 line 258, in __iter__
 self._fetch_all()
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
  
 line 1074, in _fetch_all
 self._result_cache = list(self.iterator())
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/query.py",
  
 line 158, in __iter__
 for row in compiler.results_iter():
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/sql/compiler.py",
  
 line 806, in results_iter
 results = self.execute_sql(MULTI)
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/models/sql/compiler.py",
  
 line 852, in execute_sql
 cursor.execute(sql, params)
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/backends/utils.py",
  
 line 59, in execute
 self.db.validate_no_broken_transaction()
   File 
 "/home/nemesis/.virtualenvs/djnetconfig3/lib/python3.4/site-packages/django/db/backends/base/base.py",
  
 line 429, in

Re: Referrer Policy Delivery, Django shouldn't do strict referrer check anymore?

[Collin Anderson]

Also, if we checked the origin header, would it allow us to at least
support the "Origin When Cross-Origin" policy in all browsers? (Use the
Origin header for Safari and the referrer for all of the other browsers?)


On Fri, Dec 4, 2015 at 10:38 AM, Tim Graham  wrote:

> Flávio, thanks -- since you seem to have a good understanding of the
> limitation, could you submit a documentation patch (or even just provide
> some draft text here)?
>
> On Friday, December 4, 2015 at 8:25:35 AM UTC-5, Flávio Junior wrote:
>>
>> Found a issue that already discusses this:
>> https://code.djangoproject.com/ticket/16870#comment:10
>>
>> Em quinta-feira, 3 de dezembro de 2015 13:41:09 UTC-3, Flávio Junior
>> escreveu:
>>>
>>> Florian, then Django will have to keep this limitation: can't use a
>>> global no-referrer policy on HTTPS because of strict referrer check.
>>> Correct? Should I create an issue to keep this logged?
>>>
>>> Em quinta-feira, 3 de dezembro de 2015 13:19:38 UTC-3, Florian Apolloner
>>> escreveu:



 On Wednesday, December 2, 2015 at 7:37:30 PM UTC+1, Flávio Junior wrote:
>
> If Django still needs
>  the strict
> referrer check, maybe a better error message should be implemented.
>

 I do not see any reason why it would not need it anymore. Django needs
 to ensure that the request came from a "trusted origin", therefore we still
 need it.

>>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/c0ce17a5-3f0e-4218-a2c8-cecf7da07fb9%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S4HGp3%3Dk39EE_CsmiF-Pk8gjEHqTeWXeSP8sOXdG1VmcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Annotation failure (Combining multiple aggregations)

[Paulo Maciel]

Combining multiple aggregations with annotate() will yield the wrong results 
, as multiple tables are cross 
joined. Due to the use of LEFT OUTER JOIN, duplicate records will be 
generated if some of the joined tables contain more records than the others

The Count 

 aggregate 
has a distinct parameter that may help:

q = Book.objects.annotate(Count('authors', distinct=True), Count('chapters', 
distinct=True))

Why not "distinct=True" to use in Sum?

q = Book.objects.annotate(Sum('val_a', distinct=True), Sum('val_b', 
distinct=True))

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fab53843-6eb7-49a7-9036-68e80343425f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Referrer Policy Delivery, Django shouldn't do strict referrer check anymore?

[Tim Graham]

Flávio, thanks -- since you seem to have a good understanding of the 
limitation, could you submit a documentation patch (or even just provide 
some draft text here)?

On Friday, December 4, 2015 at 8:25:35 AM UTC-5, Flávio Junior wrote:
>
> Found a issue that already discusses this: 
> https://code.djangoproject.com/ticket/16870#comment:10
>
> Em quinta-feira, 3 de dezembro de 2015 13:41:09 UTC-3, Flávio Junior 
> escreveu:
>>
>> Florian, then Django will have to keep this limitation: can't use a 
>> global no-referrer policy on HTTPS because of strict referrer check. 
>> Correct? Should I create an issue to keep this logged?
>>
>> Em quinta-feira, 3 de dezembro de 2015 13:19:38 UTC-3, Florian Apolloner 
>> escreveu:
>>>
>>>
>>>
>>> On Wednesday, December 2, 2015 at 7:37:30 PM UTC+1, Flávio Junior wrote:

 If Django still needs 
  the strict 
 referrer check, maybe a better error message should be implemented.

>>>
>>> I do not see any reason why it would not need it anymore. Django needs 
>>> to ensure that the request came from a "trusted origin", therefore we still 
>>> need it.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c0ce17a5-3f0e-4218-a2c8-cecf7da07fb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Referrer Policy Delivery, Django shouldn't do strict referrer check anymore?

[Flávio Junior]

Found a issue that already discusses 
this: https://code.djangoproject.com/ticket/16870#comment:10

Em quinta-feira, 3 de dezembro de 2015 13:41:09 UTC-3, Flávio Junior 
escreveu:
>
> Florian, then Django will have to keep this limitation: can't use a global 
> no-referrer policy on HTTPS because of strict referrer check. Correct? 
> Should I create an issue to keep this logged?
>
> Em quinta-feira, 3 de dezembro de 2015 13:19:38 UTC-3, Florian Apolloner 
> escreveu:
>>
>>
>>
>> On Wednesday, December 2, 2015 at 7:37:30 PM UTC+1, Flávio Junior wrote:
>>>
>>> If Django still needs 
>>>  the strict 
>>> referrer check, maybe a better error message should be implemented.
>>>
>>
>> I do not see any reason why it would not need it anymore. Django needs to 
>> ensure that the request came from a "trusted origin", therefore we still 
>> need it.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/066ea758-b623-4292-bc26-79ba4878f63d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.