Re: Model-level validation
I see two separate concerns here: 1) Should Django present to users the option to do validate-on-save by default? That is, should that option be visible -- in the form of a documented setting or an optional argument to save()? I tend to accept James' (and others) views and reasoning against that. 2) Can a user activate validation-on-save-by-default without resorting to monkeypatching? Should it be possible? I think applying such validation should be possible -- because, in many places, you see less-than-disciplined teams creating large projects containing heaps of code that is not of the finest quality. And I think we should help these projects improve incrementally -- that is, introduce means to improve their situation; promoting notions like data mutation should occur only in well-defined, controlled ways without making them prerequisite. This notion is a nice ideal, but installing it on a project after-the-fact is hard; in many places it is not realistically attainable -- at least in the boundaries of the team's resources, delivery requirements, and a reasonable timeframe. Note that for such general validation, a project-wide-base-model as suggested e.g. by Uri is, in general, not sufficient, because it may not apply to models from 3rd-party apps, or even from django.contrib apps. Most models in such apps are not swappable. But there is a way, I think, using a pre_save signal. One can write a small app to install pre-save validation on all models in the project, merely by including it in INSTALLED_APPS. Basically, from django.db.models import signals ... class WhateverConfig(AppConfig): ... def ready() def validate(sender, instance, raw, **kwargs): if not raw: instance.full_clean() signals.pre_save.connect(validate, weak=False) This, of course, is just a POC -- it doesn't include things like allowing a model to opt out, for example. Or one might want to apply it only where settings.DEBUG is set. Or only log warnings. Or a few other variations that don't jump to my mind immediately. But it is a way for those of us involved with large teams and projects to add this feature, without affecting the experience of newcomers or the layer-separation sensibilities of the framework. HTH, Shai. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20221010163935.4ba0fcb6.shai%40platonix.com.
Re: ArrayField: Bug on exact_nested_null
>Imho the important bit
> is, that ARRAY will drop the top level array to {} (empty array with no
> dimension info), if there is at least one empty sub array declared with
> ARRAY among NULLs, while if all sub entries are NULL the top level array
> will still manifest with dimension info, but now with entries set to
> NULL. Thats a quite weird behavior imho.
Indeed, thanks for sharing that Jörg!
Just to be sure, on postgresql I have,
psql> select version();
PostgreSQL 12.11 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 7.3.1
20180712 (Red Hat 7.3.1-12), 64-bit
psql>select (array[array[NULL,NULL]]);
{{NULL,NULL}}
psql>select array['{NULL, NULL}'] ;
{"{NULL, NULL}"}
The third is correct, but not the generated query I'd expect when writing
`nested_field=[[None, None]]` in the ORM.
(It outputs what I'd expect from `nested_field=["{NULL, NULL}"]`).
I wonder whether this bug could not be fixed in django by generating the
additional "array" in nested fields,
(I did not yet figure out how the additional array is generated when at
least one non NULL field is put there, it is in my todos :)).
WDYT?
Le lun. 10 oct. 2022 à 11:15, Jörg Breitbart
a écrit :
> Not sure if it is related, but the last sentence here caught my attention:
>
> > I wonder whether this is due to django or psycopg2. Indeed, the bug is
> > not reproduced
> > if at least one of the element in the nested array is not null.
>
>
> When doing a COPY FROM replacement of bulk_update for postgres I
> stumbled over several weird edge cases of nested arrays and its NULL
> behavior - documented here
>
> https://github.com/netzkolchose/django-fast-update/blob/59b0e6fa6affdcf8c66b2edee2e1bf1b926f4180/fast_update/copy.py#L566
>
> This seems to be a postgres internal thing and how they implemented
> ARRAY, as it is reproducible in pgsql as well. Imho the important bit
> is, that ARRAY will drop the top level array to {} (empty array with no
> dimension info), if there is at least one empty sub array declared with
> ARRAY among NULLs, while if all sub entries are NULL the top level array
> will still manifest with dimension info, but now with entries set to
> NULL. Thats a quite weird behavior imho.
>
>
> Am 09.10.22 um 12:49 schrieb Ion Alberdi:
> > Hello to all,
> >
> > a unit test added at
> > https://github.com/pricemoov/django/pull/2
> > shows that the exact filter on a full "NULL nested array" currently
> fails.
> > (the commit messages shows how to reproduce the error).
> >
> > More precisely, the error is
> > "django.db.utils.DataError: invalid input syntax for type integer:
> > "{NULL,NULL}"
> >
> > LINE 1: ...ullableintegerarraymodel"."field_nested" =
> > (ARRAY['{NULL,NUL..."
> >
> > and seems to be due to the query being generated as
> > "...ARRAY['{NULL,NULL}']..."
> > instead of "...ARRAY[ARRAY[NULL,NULL]]..."
> >
> > I wonder whether this is due to django or psycopg2. Indeed, the bug is
> > not reproduced
> > if at least one of the element in the nested array is not null.
> >
> > Is the bug worth creating a ticket?
> >
> >
> > Best regards,
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to django-developers+unsubscr...@googlegroups.com
> > .
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com
> <
> https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com?utm_medium=email_source=footer
> >.
>
> --
> netzkolchose.de UG (haftungsbeschränkt)
> Geschäftsführer: Jörg Breitbart
> Handelsregister: HRB 504791 Amtsgericht Jena
> Steuer-Nr.: 161/115/07450
> USt-IdNr.: DE268234065
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/4OvbN0EUUsM/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ca23d835-08cc-1c7a-323c-faa552560ef6%40netzkolchose.de
> .
>
--
[image: img]
*Ion Alberdi*
[image: img] [image: img]
[image: img]
[image: img]
[image: img]
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django
Re: ArrayField: Bug on exact_nested_null
Not sure if it is related, but the last sentence here caught my attention:
> I wonder whether this is due to django or psycopg2. Indeed, the bug is
> not reproduced
> if at least one of the element in the nested array is not null.
When doing a COPY FROM replacement of bulk_update for postgres I
stumbled over several weird edge cases of nested arrays and its NULL
behavior - documented here
https://github.com/netzkolchose/django-fast-update/blob/59b0e6fa6affdcf8c66b2edee2e1bf1b926f4180/fast_update/copy.py#L566
This seems to be a postgres internal thing and how they implemented
ARRAY, as it is reproducible in pgsql as well. Imho the important bit
is, that ARRAY will drop the top level array to {} (empty array with no
dimension info), if there is at least one empty sub array declared with
ARRAY among NULLs, while if all sub entries are NULL the top level array
will still manifest with dimension info, but now with entries set to
NULL. Thats a quite weird behavior imho.
Am 09.10.22 um 12:49 schrieb Ion Alberdi:
Hello to all,
a unit test added at
https://github.com/pricemoov/django/pull/2
shows that the exact filter on a full "NULL nested array" currently fails.
(the commit messages shows how to reproduce the error).
More precisely, the error is
"django.db.utils.DataError: invalid input syntax for type integer:
"{NULL,NULL}"
LINE 1: ...ullableintegerarraymodel"."field_nested" =
(ARRAY['{NULL,NUL..."
and seems to be due to the query being generated as
"...ARRAY['{NULL,NULL}']..."
instead of "...ARRAY[ARRAY[NULL,NULL]]..."
I wonder whether this is due to django or psycopg2. Indeed, the bug is
not reproduced
if at least one of the element in the nested array is not null.
Is the bug worth creating a ticket?
Best regards,
--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to django-developers+unsubscr...@googlegroups.com
.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com .
--
netzkolchose.de UG (haftungsbeschränkt)
Geschäftsführer: Jörg Breitbart
Handelsregister: HRB 504791 Amtsgericht Jena
Steuer-Nr.: 161/115/07450
USt-IdNr.: DE268234065
--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/ca23d835-08cc-1c7a-323c-faa552560ef6%40netzkolchose.de.
Re: ArrayField: Bug on exact_nested_null
No pb & crystal clear, ticket created at
https://code.djangoproject.com/ticket/34080#ticket
thanks again Adam!
Le lundi 10 octobre 2022 à 10:02:32 UTC+2, Adam Johnson a écrit :
> Ah sorry didn't realize.
>
> I would just link to the test from your ticket for now, the fellows are
> probably in a better position to determine how to treat this bug.
>
> On Mon, Oct 10, 2022 at 8:47 AM Ion Alberdi
> wrote:
>
>> Great, thanks Adam! FYI the test is already on django's Test suite
>> (https://github.com/pricemoov/django/pull/2/files)
>> Would you like me to open a PR in Django or should I wait for the fix of
>> the test being developed first?
>>
>>
>> Le lun. 10 oct. 2022 à 09:39, 'Adam Johnson' via Django developers
>> (Contributions to Django itself) a écrit :
>>
>>> Yes please create a ticket. If you’re feeling brave, try to adapt your
>>> test into Django’s test suite (here:
>>> https://github.com/django/django/blob/84206607d6bfd61e7f7a88b51163ffd4153e3b5a/tests/postgres_tests/test_array.py#L212
>>>
>>> )!
>>>
>>> On Sun, Oct 9, 2022 at 11:49 AM Ion Alberdi
>>> wrote:
>>>
Hello to all,
a unit test added at
https://github.com/pricemoov/django/pull/2
shows that the exact filter on a full "NULL nested array" currently
fails.
(the commit messages shows how to reproduce the error).
More precisely, the error is
"django.db.utils.DataError: invalid input syntax for type integer:
"{NULL,NULL}"
LINE 1: ...ullableintegerarraymodel"."field_nested" =
(ARRAY['{NULL,NUL..."
and seems to be due to the query being generated as
"...ARRAY['{NULL,NULL}']..."
instead of "...ARRAY[ARRAY[NULL,NULL]]..."
I wonder whether this is due to django or psycopg2. Indeed, the bug is
not reproduced
if at least one of the element in the nested array is not null.
Is the bug worth creating a ticket?
Best regards,
--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to django-develop...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com
.
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/django-developers/4OvbN0EUUsM/unsubscribe
>>> .
>>> To unsubscribe from this group and all its topics, send an email to
>>> django-develop...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/CAMyDDM1Rs%3DxZjChzqU3qj6YttB3atLX7bhGuURd0-3v_OitK0g%40mail.gmail.com
>>>
>>>
>>> .
>>>
>>
>>
>> --
>> [image: img]
>>
>>
>> *Ion Alberdi*
>>
>>
>> [image: img] [image: img]
>> [image: img]
>> [image: img]
>> [image: img]
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CANbgw4B81hwFvRhCVQpm%3DPg%3D_ZJikTVKKhAaXa%2B7rPpw8%3DoVEw%40mail.gmail.com
>>
>>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/54c4119f-00bf-4b46-bc57-3f56c9f37af6n%40googlegroups.com.
Re: ArrayField: Bug on exact_nested_null
Ah sorry didn't realize.
I would just link to the test from your ticket for now, the fellows are
probably in a better position to determine how to treat this bug.
On Mon, Oct 10, 2022 at 8:47 AM Ion Alberdi
wrote:
> Great, thanks Adam! FYI the test is already on django's Test suite
> (https://github.com/pricemoov/django/pull/2/files)
> Would you like me to open a PR in Django or should I wait for the fix of
> the test being developed first?
>
>
> Le lun. 10 oct. 2022 à 09:39, 'Adam Johnson' via Django developers
> (Contributions to Django itself) a
> écrit :
>
>> Yes please create a ticket. If you’re feeling brave, try to adapt your
>> test into Django’s test suite (here:
>> https://github.com/django/django/blob/84206607d6bfd61e7f7a88b51163ffd4153e3b5a/tests/postgres_tests/test_array.py#L212
>> )!
>>
>> On Sun, Oct 9, 2022 at 11:49 AM Ion Alberdi
>> wrote:
>>
>>> Hello to all,
>>>
>>> a unit test added at
>>> https://github.com/pricemoov/django/pull/2
>>> shows that the exact filter on a full "NULL nested array" currently
>>> fails.
>>> (the commit messages shows how to reproduce the error).
>>>
>>> More precisely, the error is
>>> "django.db.utils.DataError: invalid input syntax for type integer:
>>> "{NULL,NULL}"
>>>
>>> LINE 1: ...ullableintegerarraymodel"."field_nested" =
>>> (ARRAY['{NULL,NUL..."
>>>
>>> and seems to be due to the query being generated as
>>> "...ARRAY['{NULL,NULL}']..."
>>> instead of "...ARRAY[ARRAY[NULL,NULL]]..."
>>>
>>> I wonder whether this is due to django or psycopg2. Indeed, the bug is
>>> not reproduced
>>> if at least one of the element in the nested array is not null.
>>>
>>> Is the bug worth creating a ticket?
>>>
>>> Best regards,
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com
>>>
>>> .
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/django-developers/4OvbN0EUUsM/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CAMyDDM1Rs%3DxZjChzqU3qj6YttB3atLX7bhGuURd0-3v_OitK0g%40mail.gmail.com
>>
>> .
>>
>
>
> --
> [image: img]
>
>
> *Ion Alberdi*
>
>
> [image: img] [image: img]
> [image: img]
> [image: img]
> [image: img]
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CANbgw4B81hwFvRhCVQpm%3DPg%3D_ZJikTVKKhAaXa%2B7rPpw8%3DoVEw%40mail.gmail.com
>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAMyDDM19RVNqA0OiDqpwdNq0SD_uPCGr8Gz4XiWQFEqd4EZ90A%40mail.gmail.com.
Re: ArrayField: Bug on exact_nested_null
Great, thanks Adam! FYI the test is already on django's Test suite
(https://github.com/pricemoov/django/pull/2/files)
Would you like me to open a PR in Django or should I wait for the fix of
the test being developed first?
Le lun. 10 oct. 2022 à 09:39, 'Adam Johnson' via Django developers
(Contributions to Django itself) a
écrit :
> Yes please create a ticket. If you’re feeling brave, try to adapt your
> test into Django’s test suite (here:
> https://github.com/django/django/blob/84206607d6bfd61e7f7a88b51163ffd4153e3b5a/tests/postgres_tests/test_array.py#L212
> )!
>
> On Sun, Oct 9, 2022 at 11:49 AM Ion Alberdi
> wrote:
>
>> Hello to all,
>>
>> a unit test added at
>> https://github.com/pricemoov/django/pull/2
>> shows that the exact filter on a full "NULL nested array" currently fails.
>> (the commit messages shows how to reproduce the error).
>>
>> More precisely, the error is
>> "django.db.utils.DataError: invalid input syntax for type integer:
>> "{NULL,NULL}"
>>
>> LINE 1: ...ullableintegerarraymodel"."field_nested" =
>> (ARRAY['{NULL,NUL..."
>>
>> and seems to be due to the query being generated as
>> "...ARRAY['{NULL,NULL}']..."
>> instead of "...ARRAY[ARRAY[NULL,NULL]]..."
>>
>> I wonder whether this is due to django or psycopg2. Indeed, the bug is
>> not reproduced
>> if at least one of the element in the nested array is not null.
>>
>> Is the bug worth creating a ticket?
>>
>> Best regards,
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com
>>
>> .
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/4OvbN0EUUsM/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAMyDDM1Rs%3DxZjChzqU3qj6YttB3atLX7bhGuURd0-3v_OitK0g%40mail.gmail.com
>
> .
>
--
[image: img]
*Ion Alberdi*
[image: img] [image: img]
[image: img]
[image: img]
[image: img]
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CANbgw4B81hwFvRhCVQpm%3DPg%3D_ZJikTVKKhAaXa%2B7rPpw8%3DoVEw%40mail.gmail.com.
Re: ArrayField: Bug on exact_nested_null
Yes please create a ticket. If you’re feeling brave, try to adapt your test
into Django’s test suite (here:
https://github.com/django/django/blob/84206607d6bfd61e7f7a88b51163ffd4153e3b5a/tests/postgres_tests/test_array.py#L212
)!
On Sun, Oct 9, 2022 at 11:49 AM Ion Alberdi
wrote:
> Hello to all,
>
> a unit test added at
> https://github.com/pricemoov/django/pull/2
> shows that the exact filter on a full "NULL nested array" currently fails.
> (the commit messages shows how to reproduce the error).
>
> More precisely, the error is
> "django.db.utils.DataError: invalid input syntax for type integer:
> "{NULL,NULL}"
>
> LINE 1: ...ullableintegerarraymodel"."field_nested" =
> (ARRAY['{NULL,NUL..."
>
> and seems to be due to the query being generated as
> "...ARRAY['{NULL,NULL}']..."
> instead of "...ARRAY[ARRAY[NULL,NULL]]..."
>
> I wonder whether this is due to django or psycopg2. Indeed, the bug is not
> reproduced
> if at least one of the element in the nested array is not null.
>
> Is the bug worth creating a ticket?
>
> Best regards,
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/bad3435e-1299-4b1d-b701-7881e111ec90n%40googlegroups.com
>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAMyDDM1Rs%3DxZjChzqU3qj6YttB3atLX7bhGuURd0-3v_OitK0g%40mail.gmail.com.
