Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Simon Charette):

 * owner:  Simon Charette => (none)
 * status:  assigned => new


Comment:

 I fear I won't be able to work on this issue in the near future so I'm
 deassigning myself.

 Now that we filter out literal `None` values from `__in=[None]` lookup
 (#31667) we do have a mismatch with `__in=nullable_expression` but at
 least the behaviour is consistent with the
 `nullable_field=F('nullable_field')` and `nullable_field=None` asymmetry
 (#32043).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.b27cf0a96281877a75202169644b4f49%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  Simon
 |  Charette
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Simon Charette):

 * owner:  (none) => Simon Charette
 * status:  new => assigned


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.51e8fb850c6f785abd5dbdb4d5c2c724%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by felixxm):

 * has_patch:  1 => 0


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.817bde8b8d9873bd223d3d4461eacd9c%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Adam Sołtysik):

 * owner:  Adam Sołtysik => (none)
 * status:  assigned => new


Comment:

 It would be best to make the behaviour of `In` lookup consistent with
 `RelatedIn`. Sadly, I wasn't able to understand how the latter works. So
 my patch currently crashes with window functions, and it also crashes with
 `OuterRef` due to #31714. As much as I'd like to finish this, I don't have
 enough time and knowledge to look into it further. If anyone comes up with
 a solution, feel free to claim this issue.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.bf668e0ad7f3abdf3354b08ed6cdba28%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  Adam
 |  Sołtysik
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Adam Sołtysik):

 * has_patch:  0 => 1


Comment:

 I've created a
 
[https://github.com/adamsol/django/commit/f50e315ce315ead117bfe4335a9fda758e88ae6f
 patch]. No PR yet, though, because one test fails:
 `WindowFunctionTests.test_window_expression_within_subquery`. SQL doesn't
 support filtering by window functions, so I guess the code should be
 corrected to use another subquery (or maybe an `EXISTS` query).
 Unfortunately I don't know how to write this without using QuerySet
 utilities.

 As for `RelatedIn`, the things are weird here. It seems this case is
 handled differently and already works correctly, and worked even before
 #31667 (`test_ticket20024_related_in`). It was probably fixed in #13815
 long ago. There is `IS NOT NULL` in the subquery, actually there are two
 next to each other. One is coming from `split_exclude()` (called in
 `build_filter()`), but the other one -- I couldn't find. Also, what looks
 like another thing to take a closer look at -- there is `NOT IN (None)`
 instead of `NOT IN (NULL)` in the SQL for the first query in that test,
 when the `rhs.discard(None)` line is removed.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.11339f62a20dc73d3ff39538a05f49de%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  Adam
 |  Sołtysik
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Simon Charette):

 * cc: Simon Charette (added)


Comment:

 Thanks for surfacing this inconsistency Adam.

 It looks like adding an `else` clause to deal with a ''query'' right hand
 side value that has explicitly selected fields that can be null (primary
 keys can't)

 
https://github.com/django/django/blob/632ccffc49658de5348c51a8918719364be54f37/django/db/models/lookups.py#L387-L389

 The `RelatedIn` lookup will also need to be adjusted

 
https://github.com/django/django/blob/e74b3d724e5ddfef96d1d66bd1c58e7aae26fc85/django/db/models/fields/related_lookups.py#L98-L99

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.8feb69ead65407e79f28f29d610d9ca8%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  Adam
 |  Sołtysik
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Adam Sołtysik):

 * owner:  nobody => Adam Sołtysik
 * status:  new => assigned
 * version:  1.4 => master


Comment:

 I wanted to fix this, and I've found that it has already been partially
 fixed in #31667. However, there are no tests for `exclude`, and the
 behaviour is now inconsistent, since
 `exclude(col__in=queryset.values('col2'))` still gives no results when
 `col2` contains `NULL`. I'll try to provide a patch to clean this up.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.fb2e89042cdf045aafcf6d357e9b3cd2%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Adam Sołtysik):

 I think this should be fixed, this a seriously surprising flaw in the ORM.
 I've proposed several possible fixes for this issue in #31883, I'll try to
 summarize them here.

 The simplest would be to just ignore `None`s in the list. This would be
 consistent with how `filter` currently works. When dealing with querysets
 (`__in=qs.values('col')`), it should be enough to add `WHERE col IS NOT
 NULL` in the `SELECT` subquery.

 A better solution, but a more breaking change, would be to add a proper
 `IS [NOT] NULL` check, in both `exclude` and `filter` cases, if `None` is
 in the list. This would allow writing `filter(col__in=[something, None])`
 instead of `filter(Q(col=something) | Q(col=None))`. Some additional
 checks for inner querysets would also be required, so that the behaviour
 is consistent, or the queryset could be just converted to a list
 beforehand.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.dda467946abb30b4e348a182e06dfd47%40djangoproject.com.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element.

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Carsten Fuchs):

 * cc: Carsten Fuchs (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.942936b23d14611d06d123f9e8476c0c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #20024: QuerySet.exclude() does not work with lists containing a 'None' element. (was: 'exclude' does not work with lists containing a 'None' element.)

[Django]

#20024: QuerySet.exclude() does not work with lists containing a 'None' element.
-+-
 Reporter:  stillwater.ke@…  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/081.ce0fca5ccf6895d8f6a4f5b64069dd96%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.