How would the strategy of having a setting generate new migrations work
with third-party apps? Generally migrations can't easily be added to them
by user projects and if more migrations are added to the third party app in
a later version, there will be an inconsistent history.
On Thursday, April 9, 2020 at 3:06:51 PM UTC-4, Collin Anderson wrote:
>
> Having a DEFAULT_AUTOFIELD setting makes sense to me.
>
> On Monday, April 6, 2020 at 12:48:10 PM UTC-4, Adam Johnson wrote:
>>
>> Jure - yes switching the setting should generate migrations for all
>> affected models. The migration guide would cover changing models' primary
>> key fields to PositiveBigAutoFields one at a time, perhaps with a mixin as
>> you've suggested. Maybe Django should provide such a mixin to ease the
>> migration.
>>
>> primary keys might be GUID
>>>
>>
>> The proposal is not to move to GUID's/UUID's, as you've used in your
>> example. It's to move to bigger integers.
>>
>> UUID's are a bad idea for database performance, as they distribute
>> randomly across the table, preventing any cache wins. On autoincrement
>> tables, the tail end of each table is normally most frequently read and
>> written and thus cached in memory. I don't think Django should ever suggest
>> UUID's as primary keys.
>>
>> On Mon, 6 Apr 2020 at 16:46, Jure Erznožnik <jure.e...@gmail.com> wrote:
>>
>>> Sorry if I understand stuff incorrectly, but:
>>>
>>> I agree with Adam where he discusses migration issues, but I also don't
>>> see how "DEFAULT_AUTOFIELD" setting could leave tables out of the
>>> migration(s).
>>>
>>> My understanding is that it would cause immediate re-provisioning of all
>>> the pk fields, including in the core django modules and third-party
>>> modules. Some of them might have programmed their logic such as to include
>>> some integer math around the primary keys (and now all auto primary keys
>>> might be GUID).
>>>
>>> If I understand the above correctly, the setting would have to be
>>> per-module. Could it simply be done like this:
>>>
>>> class PKMixin(object): id = models.UUIDField(primary_key=True, ....)
>>>
>>> Naturally, then all desired models would (also) derive from this mixin.
>>> A bit of a chore, but it would not require reconfiguration of all
>>> migrations, just the ones the programmer would specify - and would only be
>>> limited to their own module.
>>>
>>> OTOH, JUST FOR the unsigned vs signed integer PK, it should be
>>> relatively easy to modify makemigrations code detect existing migrations
>>> for the model, detect that the only difference is IntegerField vs
>>> PositiveIntegerField and in this case skip generating the migration (for
>>> migrating the field to an unsigned one). There could also be a flag to the
>>> management command specifying to force generating said migrations.
>>>
>>> Of course, ignore if this is gibberish.
>>>
>>> LP,
>>> Jure
>>>
>>>
>>> On 06/04/2020 17:11, Adam Johnson wrote:
>>>
>>> I'm in favour of the move. The setting strategy seems appropriate.
>>>
>>> Simon's comment on the PR that this affects a small % of projects is
>>> true. But if your project does reach that scale, you're probably one of the
>>> bigger Django organizations in the world - hopefully key advocates for the
>>> framework. It's also possible to hit this on smaller projects with tables
>>> that import a lot of new rows, such as rolling time series data imports.
>>>
>>> Also, this problem is *highly* asymmetric. Unnecessarily using big PK
>>> fields wastes a little storage space - unlikely to be noticed. Migrating to
>>> big PK fields can be a massive task (and can feel like "Django let you down
>>> here with bad defaults") ("Ruby on Rails does it right!").
>>>
>>> It will need a careful migration guide though. Pre-existing projects may
>>> need to migrate one table at a time to reduce downtime/load, or keep the
>>> setting to the smaller auto field forever. I'm happy to help with this.
>>>
>>> One thing we *could* add is a deploy time system check (or a management
>>> command, or something), to check what % of your tables' autofields' PK
>>> values have been "used up." This allows larger projects to prioritize their
>>> migrations.
>>>
>>> On Mon, 6 Apr 2020 at 15:37, Caio Ariede <caio....@gmail.com> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> I’ve been working on ticket #56 "Primary key columns should be
>>>> UNSIGNED" (really old) for a while now. It’s cumbersome and hard to
>>>> fix for some reasons, including backwards compatibility, nothing that a
>>>> single PR can solve but I’ll try to summarize where things are at this
>>>> moment. Hopefully we can get to a consensus so I can continue.
>>>>
>>>> The problem: Django has been storing primary keys as signed integers
>>>> since the beginning and this is considered a waste of resources. Very few
>>>> people seem to use negative values as primary keys, apparently.
>>>>
>>>> The desired solution: We want to change that in a backwards-compatible
>>>> way to not cause migrations to happen in all projects using Django, in all
>>>> of a sudden. This is where things start to get difficult.
>>>>
>>>> These are the links for the related ticket and PR up to this point:
>>>>
>>>> https://code.djangoproject.com/ticket/56 (Primary key columns should
>>>> be UNSIGNED)
>>>> https://github.com/django/django/pull/11900 (PR)
>>>>
>>>> —
>>>>
>>>> While I was working on PR 11900, I stumbled across this other PR 8924
>>>> "BigAutoField as new default in 2.0”, and a particular comment from Nick
>>>> Pope got my attention:
>>>>
>>>> https://github.com/django/django/pull/8924#issuecomment-516792989
>>>>
>>>> The idea o adding a new DEFAULT_AUTOFIELD setting made a lot of sense
>>>> to me, as we would be able to keep things backwards compatible.
>>>>
>>>> My first reaction after following that discussion, was to add a missing
>>>> part that would likely help, which was the PositiveBigIntegerField. This
>>>> is
>>>> already fixed and merged:
>>>>
>>>> https://code.djangoproject.com/ticket/30987
>>>>
>>>> —
>>>>
>>>> Now that we have PositiveBigIntegerField merged in, we can also have
>>>> PositiveBigAutoField
>>>> and finally move forward with ticket #56. But we’d still need the ability
>>>> to set a different default Auto Field.
>>>>
>>>> To achieve this, I’ve created a new ticket and Mariusz Felisiak
>>>> suggested to bring this discussion to the developers list, whether or not
>>>> we should move forward with this approach:
>>>>
>>>> https://code.djangoproject.com/ticket/31007
>>>>
>>>> I want to hear from you folks, whether or not we can get to a consensus
>>>> on adding this new setting but also, hear any concerns or issues you can
>>>> anticipate.
>>>>
>>>> Ideally, this new setting would default to AutoField until Django 4?
>>>> After that we can make it default to PositiveBigAutoField.
>>>>
>>>> We could (or should) also change the project template to set
>>>> DEFAULT_AUTOFIELD
>>>> to PositiveBigAutoField for new projecst, as suggested in Nick Pope's
>>>> comment above.
>>>>
>>>>
>>>> Thank you
>>>>
>>>> --
>>>> Caio
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Django developers (Contributions to Django itself)" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to django-d...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/django-developers/8E947B86-2228-4DD3-9D6A-C1B6D757652E%40gmail.com
>>>>
>>>> <https://groups.google.com/d/msgid/django-developers/8E947B86-2228-4DD3-9D6A-C1B6D757652E%40gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>
>>>
>>> --
>>> Adam
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-d...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/CAMyDDM0VqXrPbre-6YXSSnu2PBStQoijjoEH_9Wc-aB0gDEHBw%40mail.gmail.com
>>>
>>> <https://groups.google.com/d/msgid/django-developers/CAMyDDM0VqXrPbre-6YXSSnu2PBStQoijjoEH_9Wc-aB0gDEHBw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-d...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/69378e21-2b2c-2604-18c8-16c6efc03a11%40gmail.com
>>>
>>> <https://groups.google.com/d/msgid/django-developers/69378e21-2b2c-2604-18c8-16c6efc03a11%40gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>
>> --
>> Adam
>>
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/a24eb118-ee0b-42dc-afb4-84b3e218ff62%40googlegroups.com.
