Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Nick Pope
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"34d6bceec46c5d4234c156ed682573d2e5de474a" 34d6bce]:
 {{{
 #!CommitTicketReference repository=""
 revision="34d6bceec46c5d4234c156ed682573d2e5de474a"
 Fixed #29500 -- Fixed SQLite function crashes on null values.

 Co-authored-by: Srinivas Reddy Thatiparthy
 
 Co-authored-by: Nick Pope 
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.2333e3363ee35d422802b60d485d65d2%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Nick Pope
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Nick Pope):

 * needs_better_patch:  1 => 0


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.7eab50e126e936c9a75feb8aee2d082e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Nick Pope
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * needs_better_patch:  0 => 1


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.7bd19021b18abd5cdf6602b55cfb7c43%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Nick Pope
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Nick Pope):

 * owner:  (none) => Nick Pope
 * status:  new => assigned
 * has_patch:  0 => 1


Comment:

 [https://github.com/django/django/pull/10301 PR]

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.92ad3d5aefe90a01aa96e905b74ad46e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Jeff):

 * owner:  Jeff => (none)
 * status:  assigned => new


Comment:

 Sensing a pattern. Deassigning.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.ce56707be40b2f55b8ad7ab165c39c4a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Jeff
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Jeff):

 * owner:  (none) => Jeff
 * status:  new => assigned


Comment:

 I'll be helping Srinivas get this across the finish line.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.0573967349af65b4fb886df8f68621ab%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Srinivas Reddy Thatiparthy):

 * owner:  Srinivas Reddy Thatiparthy => (none)
 * status:  assigned => new


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.8eecfc451f88fed5c2b74c8b0b015d9b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Srinivas
 |  Reddy Thatiparthy
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Nick Pope):

 We need to be careful how we handle this to ensure that the behaviour
 mirrors other backends.

 I've checked PostgreSQL and when any one of the arguments to `POWER()`.
 `LPAD()` or `RPAD()` is `NULL` they return `NULL`.
 We should ensure that we check whether any one of the arguments is `None`
 and, if so, return `None`.

 We must not catch exceptions such as `TypeError` or `ValueError` to do
 this as has been done in the initial version of the
 [https://github.com/django/django/pull/10121 PR].
 If we were to pass a string to `_sqlite_power()` we would expect a
 `TypeError` which should blow up, not return `None`. Compare to
 PostgreSQL:
 {{{
 postgres=# select power(2, 'abc');
 ERROR:  invalid input syntax for type double precision: "abc"
 LINE 1: select power(2, 'abc');
 ^
 }}}
 The second part of the problem here is that the sqlite backend suppresses
 the error message and returns a different exception:
 {{{
 OperationalError: user-defined function raised exception
 }}}
 Obviously this is not particularly helpful, but a quick search and I found
 the following on Stack Overflow: https://stackoverflow.com/a/45834923
 It points to the documentation for
 
[https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks
 sqlite3.enable_callback_tracebacks()].

 I would recommend the following:

 1. Creation of a decorator to check for `None` passed into any of the
 arguments which returns `None` or calls the function as appropriate.
 2. Enabling callbacks on tracebacks for sqlite3 (always / when debug
 enabled / documentation change to give instruction).

 Note that the outcome of this pull request will affect
 [https://github.com/django/django/pull/9622 PR/9622] which I am reviewing,
 particular with respect to my
 [https://github.com/django/django/pull/9622#discussion_r165360894
 comment].

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.c9bbbdb14e946e7b83c543e2211fecab%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  Srinivas
 |  Reddy Thatiparthy
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Srinivas Reddy Thatiparthy):

 * owner:  nobody => Srinivas Reddy Thatiparthy
 * status:  new => assigned


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.5b3c7ee46bf78109e9eca34025a137cb%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Carlton Gibson):

 * stage:  Unreviewed => Accepted


Comment:

 OK, thanks for the clarification. We could certainly consider a PR adding
 test cases and a bulletproofing to the functions we're shipping.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.cbe1fcacb6a08f11ce1e7a9bb3eaf28e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Sergey Fedoseev):

 Replying to [comment:1 Carlton Gibson]:

 > I'll guess it'll be this:

 It is.

 By `SQLite functions` I meant user-defined function created here:
 
https://github.com/django/django/blob/6dd4edb1b4f5441c5f543e29395039839c50d10b/django/db/backends/sqlite3/base.py#L158-L175

 The list (incomplete?) of functions that crash on NULL values:
 *
 
[https://github.com/django/django/blob/6dd4edb1b4f5441c5f543e29395039839c50d10b/django/db/backends/sqlite3/base.py#L449
 _sqlite_time_diff]
 *
 
[https://github.com/django/django/blob/6dd4edb1b4f5441c5f543e29395039839c50d10b/django/db/backends/sqlite3/base.py#L464
 _sqlite_timestamp_diff]
 *
 
[https://github.com/django/django/blob/6dd4edb1b4f5441c5f543e29395039839c50d10b/django/db/backends/sqlite3/base.py#L474
 _sqlite_lpad]
 *
 
[https://github.com/django/django/blob/6dd4edb1b4f5441c5f543e29395039839c50d10b/django/db/backends/sqlite3/base.py#L480
 _sqlite_rpad]
 *
 
[https://github.com/django/django/blob/6dd4edb1b4f5441c5f543e29395039839c50d10b/django/db/backends/sqlite3/base.py#L484
 _sqlite_power]

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.af31cec96a5b633014d225a3cfe78a65%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Carlton Gibson):

 * cc: Carlton Gibson (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.e421764a2b8e51c7e32f13480aaf8478%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
 Reporter:  Sergey Fedoseev  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  2.0
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Carlton Gibson):

 Hmmm. Not sure we'll be able to do anything about this. (Postgres
 certainly behaves better.)

 Could you
 
[https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks
 enable callback trackbacks] on the client? We can then see the error.

 I'll guess it'll be this:

 ```
 >>> None ** None
 Traceback (most recent call last):
   File "", line 1, in 
 TypeError: unsupported operand type(s) for ** or pow(): 'NoneType' and
 'NoneType'
 ```

 If so we may just have to workaround it by using a function for `pow`
 which checks for `None`.

 This works:

 ```
 >>>
 
TestModel.objects.annotate(null=Value(None,output_field=models.IntegerField())).values(pow=models.F('null')).first()
 {'pow': None}
 ```

 So it's just the `**` operation.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/068.8ed48a8299a1bdfc18703db3f6ed5cff%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

[Django] #29500: SQLite functions crashes on NULL values

[Django]

#29500: SQLite functions crashes on NULL values
-+-
   Reporter:  Sergey |  Owner:  nobody
  Fedoseev   |
   Type:  Bug| Status:  new
  Component:  Database   |Version:  2.0
  layer (models, ORM)|
   Severity:  Normal |   Keywords:
   Triage Stage: |  Has patch:  0
  Unreviewed |
Needs documentation:  0  |Needs tests:  0
Patch needs improvement:  0  |  Easy pickings:  0
  UI/UX:  0  |
-+-
 {{{
 In [14]: TestModel2.objects.annotate(null=models.Value(None,
 output_field=models.IntegerField())).values(pow=models.F('null') **
 models.F('null')).first()

 ---
 OperationalError  Traceback (most recent call
 last)
 ~/dev/django/django/db/backends/utils.py in _execute(self, sql, params,
 *ignored_wrapper_args)
  84 else:
 ---> 85 return self.cursor.execute(sql, params)
  86

 ~/dev/django/django/db/backends/sqlite3/base.py in execute(self, query,
 params)
 295 query = self.convert_query(query)
 --> 296 return Database.Cursor.execute(self, query, params)
 297

 OperationalError: user-defined function raised exception
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/053.081ad3ed372077521bed3def91422bf7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.