Re: Default to BigAutoField
However, it should also be noted that those same larger applications are the ones that are likely to run into this problem eventually, so perhaps forcing the migration is the best path moving forward. Interested in hearing thoughts about this. -- Kenneth Reitz > On Jun 9, 2017, at 3:54 PM, Kenneth Reitz <kenn...@heroku.com> wrote: > > My initial thought was to just have this apply to *new* Django applications, > if that's possible. > > Running this migration could take quite some time on some larger apps, and > would require a non-trivial amount of time to execute. > > -- > Kenneth Reitz > >> On Jun 9, 2017, at 3:53 PM, Tom Forbes <t...@tomforb.es >> <mailto:t...@tomforb.es>> wrote: >> >> How would this work with generic foreign keys as well? Those migrations >> couldn't be automatic. >> >> On 9 Jun 2017 20:43, "Jacob Kaplan-Moss" <ja...@jacobian.org >> <mailto:ja...@jacobian.org>> wrote: >> I think this would be a good improvement, and I'd like to see it. I've been >> bitten by integers overflowing at least twice I can remember in my career, >> which is two times too many. >> >> However, a major thing we'd have to work out is the upgrade path Consider a >> simple model: >> >> class Person(Model): >> name = CharField() >> >> In Django 1.11, this actually generates a model with an integer `id` field. >> But in we change it, in Django vNext, that `id` field would "turn into" a >> bigint magically without the underlying table changes. That'd be confusing: >> you'd expect the model to be "fixed" by pugrading to vNext, but it wouldn't >> be. I think the migrations engine would detect this as a migration (?), so >> perhaps that's the path forward, but it could still be super-confusing. >> We've never shipped a release of Django that required a migration to _all_ >> your models. >> >> Have you thought about what the upgrade path should look like, Kenneth? >> >> Jacob >> >> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz <kenn...@heroku.com >> <mailto:kenn...@heroku.com>> wrote: >> Dear Django Dev, >> >> At Heroku, we have the privilege of seeing an extremely broad range of >> customers utilizing tools like Django to build their applications and >> companies. One of the things that we’ve seen customers hit, time and time >> again when using tools like Django, is integer overflows for primary keys. >> Their application starts behaving unpredictably once they reach the >> overflow, not even knowing such a constraint exists, and they often think >> the problem is with their database provider, rather than with their schema. >> Once they realize what is wrong, it’s a relatively trivial fix, but a >> migration can take several hours to complete, which results in unacceptable >> amounts of downtime. >> >> Because of this, Heroku, as a company, would like to encourage bigints as a >> sane reasonable default for primary keys for application models. If the >> Django project agrees with this idea, that would mean that Django would >> provide BigAutoField as the default for ‘id’ instead of AutoField. >> >> Rails made this change recently >> <http://www.mccartie.com/2016/12/05/rails-5.1.html>, and has seen success in >> doing so. >> >> I’m happy to provide the code to do this, but I wanted to discuss it here >> before doing so, to hear what the general consensus was to the proposal of >> such a change. >> >> >> Pros: >> Users of Django, following the docs, won’t accidentally hit the int overflow >> barrier. >> Encourages best-practices from the beginning. >> Bigints don’t take up much more storage than ints when using Postgres. >> In-line with other frameworks moving forward on this issue, like Rails >> <http://www.mccartie.com/2016/12/05/rails-5.1.html>. >> >> Cons: >> Backwards compatibility would need to be considered. >> >> Why not UUID? >> >> I agree! I love using UUID for my primary keys, and I think a patch to >> Django which provides an AutoUUIDField would be wonderful. However, there >> are a few major drawbacks to making this the new default: >> >> It’s confusing to new users, would make onboarding process more difficult. >> UUID is difficult to implement in MySQL. >> UUID has larger storage requirements. >> Incrementing IDs are actually useful. >> >> >> So, my proposal is to simply lift the int barrier t
Re: Default to BigAutoField
My initial thought was to just have this apply to *new* Django applications, if that's possible. Running this migration could take quite some time on some larger apps, and would require a non-trivial amount of time to execute. -- Kenneth Reitz > On Jun 9, 2017, at 3:53 PM, Tom Forbes <t...@tomforb.es> wrote: > > How would this work with generic foreign keys as well? Those migrations > couldn't be automatic. > > On 9 Jun 2017 20:43, "Jacob Kaplan-Moss" <ja...@jacobian.org > <mailto:ja...@jacobian.org>> wrote: > I think this would be a good improvement, and I'd like to see it. I've been > bitten by integers overflowing at least twice I can remember in my career, > which is two times too many. > > However, a major thing we'd have to work out is the upgrade path Consider a > simple model: > > class Person(Model): > name = CharField() > > In Django 1.11, this actually generates a model with an integer `id` field. > But in we change it, in Django vNext, that `id` field would "turn into" a > bigint magically without the underlying table changes. That'd be confusing: > you'd expect the model to be "fixed" by pugrading to vNext, but it wouldn't > be. I think the migrations engine would detect this as a migration (?), so > perhaps that's the path forward, but it could still be super-confusing. We've > never shipped a release of Django that required a migration to _all_ your > models. > > Have you thought about what the upgrade path should look like, Kenneth? > > Jacob > > On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz <kenn...@heroku.com > <mailto:kenn...@heroku.com>> wrote: > Dear Django Dev, > > At Heroku, we have the privilege of seeing an extremely broad range of > customers utilizing tools like Django to build their applications and > companies. One of the things that we’ve seen customers hit, time and time > again when using tools like Django, is integer overflows for primary keys. > Their application starts behaving unpredictably once they reach the overflow, > not even knowing such a constraint exists, and they often think the problem > is with their database provider, rather than with their schema. Once they > realize what is wrong, it’s a relatively trivial fix, but a migration can > take several hours to complete, which results in unacceptable amounts of > downtime. > > Because of this, Heroku, as a company, would like to encourage bigints as a > sane reasonable default for primary keys for application models. If the > Django project agrees with this idea, that would mean that Django would > provide BigAutoField as the default for ‘id’ instead of AutoField. > > Rails made this change recently > <http://www.mccartie.com/2016/12/05/rails-5.1.html>, and has seen success in > doing so. > > I’m happy to provide the code to do this, but I wanted to discuss it here > before doing so, to hear what the general consensus was to the proposal of > such a change. > > > Pros: > Users of Django, following the docs, won’t accidentally hit the int overflow > barrier. > Encourages best-practices from the beginning. > Bigints don’t take up much more storage than ints when using Postgres. > In-line with other frameworks moving forward on this issue, like Rails > <http://www.mccartie.com/2016/12/05/rails-5.1.html>. > > Cons: > Backwards compatibility would need to be considered. > > Why not UUID? > > I agree! I love using UUID for my primary keys, and I think a patch to Django > which provides an AutoUUIDField would be wonderful. However, there are a few > major drawbacks to making this the new default: > > It’s confusing to new users, would make onboarding process more difficult. > UUID is difficult to implement in MySQL. > UUID has larger storage requirements. > Incrementing IDs are actually useful. > > > So, my proposal is to simply lift the int barrier to a bigint barrier for new > Django applications, and I think it will save a lot of developers a lot of > pain in the long run. > > Many thanks, > > Kenneth Reitz > Heroku Python > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com > <mailto:django-developers+unsubscr...@googlegroups.com>. > To post to this group, send email to django-developers@googlegroups.com > <mailto:django-developers@googlegroups.com>. > Visit this group at https://groups.google
Default to BigAutoField
Dear Django Dev, At Heroku, we have the privilege of seeing an extremely broad range of customers utilizing tools like Django to build their applications and companies. One of the things that we’ve seen customers hit, time and time again when using tools like Django, is integer overflows for primary keys. Their application starts behaving unpredictably once they reach the overflow, not even knowing such a constraint exists, and they often think the problem is with their database provider, rather than with their schema. Once they realize what is wrong, it’s a relatively trivial fix, but a migration can take several hours to complete, which results in unacceptable amounts of downtime. Because of this, Heroku, as a company, would like to encourage bigints as a sane reasonable default for primary keys for application models. If the Django project agrees with this idea, that would mean that Django would provide BigAutoField as the default for ‘id’ instead of AutoField. Rails made this change recently <http://www.mccartie.com/2016/12/05/rails-5.1.html>, and has seen success in doing so. I’m happy to provide the code to do this, but I wanted to discuss it here before doing so, to hear what the general consensus was to the proposal of such a change. Pros: - Users of Django, following the docs, won’t accidentally hit the int overflow barrier. - Encourages best-practices from the beginning. - Bigints don’t take up much more storage than ints when using Postgres. - In-line with other frameworks moving forward on this issue, like Rails <http://www.mccartie.com/2016/12/05/rails-5.1.html>. Cons: - Backwards compatibility would need to be considered. Why not UUID? I agree! I love using UUID for my primary keys, and I think a patch to Django which provides an AutoUUIDField would be wonderful. However, there are a few major drawbacks to making this the new default: 1. It’s confusing to new users, would make onboarding process more difficult. 2. UUID is difficult to implement in MySQL. 3. UUID has larger storage requirements. 4. Incrementing IDs are actually useful. So, my proposal is to simply lift the int barrier to a bigint barrier for new Django applications, and I think it will save a lot of developers a lot of pain in the long run. Many thanks, Kenneth Reitz Heroku Python -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/6fe3401c-4404-4bd8-9d22-58df95cd1348%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Integrate dj-database-url into Django
My thoughts are the same as Shai's below.
On Monday, May 29, 2017 at 4:00:09 PM UTC-4, Shai Berger wrote:
>
> If I understand things correctly, registration of new url-schemas to
> enable
> something like dj-database-url to support 3rd parties should be a trivial
> matter; just add an entry to some dictionary, with the schema as key, and
> the
> dotted-path of the backend as value.
>
> IMO, scanning INSTALLED_APPS, or relying on ready(), is a lot of over-
> engineering for this. Just do the registration from the settings file,
> before
> invoking the url-parsing mechanism:
>
> from django.database_url import register_backend, configure_db
>
> register_backend('pyodbc-azure', 'sql_server.pyodbc')
>
> DATABASES = {'default' : configure_db() }
>
> The only barrier I see is namespacing -- the functions mentioned above
> cannot
> live in django.conf (would cause a circular import if imported from the
> settings module) nor django.db (which shouldn't be imported too early),
> and
> that's why I wrote `django.database_url` above.
>
> What am I missing?
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/065601c3-9488-42aa-8b23-c7f906c26b5a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Integrate dj-database-url into Django
Ah, I see what you're saying about third-party backends. I imagine that you'd have to be able to merge a url string with a settings dict (one overrides the other). This is how usage of that functionality is manually provided in dj-database-url. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9043e51f-843d-4395-9cb1-9ef138108eb7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Integrate dj-database-url into Django
I don't think the code will be able to be re-used as much as the idea/ux/functionality that it provides. I can imagine any backend being able to convert from a url string with some methods (e.g. `detect_url` (bool return) and `parse_url` (settings dict return type).) Perhaps the same scheme could be applied to caches as well. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/aba85896-0271-4358-aba4-c2e4cab186b1%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Integrate dj-database-url into Django
dj-database-url gives Django developers two main things: 1. the ability to represent their database settings via a string (a-la sqlalchemy); very useful for environmnent variables and 12factor apps. 2. it will automatically use the DATABASE_URL environment variable, if present. I'm not sure if #2 is appropriate for Django Core (but it would be nice, as Gunicorn supports both PORT and WEB_CONCURRENCY), but I know #1 is perfect. This plan has previously been discussed (at a conference, DjangoCon EU in Zurich, long ago) and approved by JKM, before his role at Heroku, if that is helpful information. I think this change would vastly improve the usability of Django, and would be an excellent and simple move for the project. Many thanks for your consideration. <3 -- Kenneth Reitz https://code.djangoproject.com/ticket/28236 -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/057ced98-4471-4939-960b-900ec39f54b7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Ad-hoc Django integration for fault-tolerance
Enjoy: http://guide.couchdb.org/draft/why.html -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/vQegCKmfVvAJ. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
