> On that note, I'm kind of worried about your assertion that types in contrib libraries aren't required to support automatic migration. Since migrations were added, we've added support for migrations in most of our custom field types.
To clarify, custom fields provided in contrib.postgres are autodetected and deconstructed appropriately. However notably the hstore field requires an extension to be installed in the database. It turned out to be extremely invasive to autodetect when this extension was needed and when it wasn't needed and it would have required probably hundreds of lines of extra complexity in the autodetector, as well as defining a formal api for "database extensions", whatever that means. Enums in postgres (custom types) would fall into the same category. As a result, the decision was taken to provide Operation subclasses appropriate for anything needed, but require users to manually add that operation to a migration where necessary and ensure the dependency tree works. Similarly and extension is provided if you want to use the unaccent lookup. See https://docs.djangoproject.com/en/dev/ref/contrib/postgres/operations/. If contrib.postgres was to gain an EnumField implementation, I would expect to see the addition of a CreateEnum operation and an AlterEnum operation, but I wouldn't expect those to necessarily be autodetected. If they were, that's great, but from my research into trying to do something similar, doing it correctly while maintaining the API boundaries between AutoDetector and SchemaEditor is likely to be an extremely complex patch, in my mind for very little gain. Especially when talking about postgres, the error messages from such a failed migration should be fairly easy to understand: "type hstore does not exist", "type my_enum does not exist", so I don't think it's a big deal to expect users to manage it by hand. On 26 February 2015 at 19:38, charettes <charett...@gmail.com> wrote: > > I try to keep myself out of philosophical arguments. Reference tables >> have a purpose (when you're dealing with a set of values that aren't >> fully known when you're defining the dataset (eg. custom application >> error code tables)), but when the dataset _is_ known in advance an >> enum will save you a couple of joins per table lookup. >> > > You can avoid those couple of joins by making your referenced table > primary key what your enum value would have been and simple not joining it. > > In this case you would simply use your referenced table for data integrity > through foreign constraint enforcement. > > Simon > > Le jeudi 26 février 2015 07:38:17 UTC-5, Thomas Stephenson a écrit : >> >> Some general responses to your points: >> >> Database support >> Yep, not all databases support enums and not all support them the same >> way. That's par for the course when you're trying to abstract across >> multiple implementations of the same pseudo-standard. >> >> Vendor lock-in >> Enums are hardly vendor lock-in, they're a language feature that >> happens to require a library in order to support backwards >> compatibility. I accept the fact that there may be changes to it in >> the near future, but even if there are, there can't be any breaking >> changes to the API. >> >> Ordering >> Yes you can't do >> sorted(qs, key=lambda o: o.my_enum_field) >> But you can do >> sorted(qs, key=lambda o: o.my_enum_field.name) >> which will give you the same result as the database ordering, or >> sorted(qs, key=lambda o: o.my_enum_field.value), >> if you want to sort according to the values. >> >> Migrations >> Yep, a bit of a pain in the neck to support changing enum definitions, >> but in general any _sane_ developer would only ever add values to an >> enum after it was initially created, and you could make >> `AddValueToEnum` a non-reversible operation. >> >> I admit it does create some expectations about other places that >> migrations would require more "intelligence", but extensions to the >> original implementation are an inevitability and I'm sure you're quite >> aware of that. >> >> On that note, I'm kind of worried about your assertion that types in >> contrib libraries aren't required to support automatic migration. >> Since migrations were added, we've added support for migrations in >> most of our custom field types. >> >> Philosophy >> I try to keep myself out of philosophical arguments. Reference tables >> have a purpose (when you're dealing with a set of values that aren't >> fully known when you're defining the dataset (eg. custom application >> error code tables)), but when the dataset _is_ known in advance an >> enum will save you a couple of joins per table lookup. >> >> >> It depends what you guys want to do. I'm happy to put in the work to >> make the implementation generic, but I'm not going to push for you >> guys to implement something you don't want or don't think provides >> real value for users. Contributing to contrib.postgres is a possible >> option, but it's not really a postgres specific feature -- almost all >> of the major database vendors support it (sqlite being as always the >> obvious exception). >> >> Thomas >> >> On 26 February 2015 at 21:26, Marc Tamlyn <marc....@gmail.com> wrote: >> > I kinda like the idea of enum fields, but I feel they are likely better >> in >> > theory than in practice. In theory I said I would introduce one as part >> of >> > contrib.postgres, but I've been putting it off as I'm unconvinced >> whether it >> > is necessarily ideal anyway when compared to choices or reference >> tables. >> > >> > Database support: PG, MySQL and Oracle all have enum data types. >> However >> > postgres does treat them somewhat differently, requiring you to >> explicitly >> > create a new type allowing the same enum type to be used across >> multiple >> > tables. Which paradigm should we follow in migrations? >> > >> > Python support: This isn't a major issue as some other "core" features >> like >> > timezones require third party packages (pytz), but we are still >> requiring an >> > external package for python < 3.4. SAY NO TO VENDORING! It's also worth >> > noting it's a relatively new python level concept and there may be >> changes >> > to it. >> > >> > Ordering: I'm a little uncomfortable with the idea that you can do >> > .order_by('my_enum_field') but you can't do sorted(qs, key=lambda o: >> > o.my_enum_field), unless you use IntEnum, which the docs say you >> shouldn't. >> > >> > Migration issues: Postgres does not support removing values from enum >> > fields, although has good support for adding values. I'm not sure >> Oracle >> > supports changing enums at all, and on MySQL changing the enum is an >> ill >> > defined process with some unexpected consequences (inevitably...), >> which is >> > also extremely slow. Obviously we have no such guarantee about anything >> when >> > using choices at the moment, however with an automatic migration system >> > associated to enums developers are likely to assume more intelligence >> than >> > is present. We go to great pains in db.migrations at the moment to >> ensure >> > this. >> > >> > Philosophy: Like with choices, there are arguments against using enum >> at all >> > in favour of using reference tables. The most obvious of these being >> the >> > ability to add extra information to a reference table. On the other >> side >> > however, by allowing the right hand side of the enum in python to be >> > arbitrary objects (classes or something) then you can nicely >> encapsulate the >> > a bunch of extra information. For example consider a competition model >> where >> > there 3 or 4 different ways of working out the winner, then having an >> enum >> > with classes where you can do competition.winner_type.get_winner(). >> This >> > functional call may need additional context. >> > >> > Overall, I'm hovering around a -0 on a general implementation of >> EnumField >> > or similar. However, I'm +0 on either a) a good third party >> implementation >> > or b) a contrib.postgres specific implementation with well documented >> > caveats. An advantage of living in contrib.postgres is that you don't >> need >> > to concern yourself with automatic migrations, and contrib can have a >> more >> > lenient policy on external packages. You also get a guaranteed review >> (me!). >> > >> > Marc >> > >> > On 26 February 2015 at 02:53, Thomas Stephenson <ova...@gmail.com> >> wrote: >> >> >> >> As discussed in Issue 24342, I've got an implementation of EnumField >> that >> >> we've found useful when developing our django REST API that I'd like >> to add >> >> to django core. It was suggested I bring this issue up in the >> developers >> >> mailing list for discussion. Unfortunately, updates to the issue were >> >> delivered to my spam folder, so there has been some delay in actually >> >> raising the issue. >> >> >> >> Basically, the implementation consists of a new field type, and a new >> >> migration operation (to register the values associated with an enum >> type). >> >> The field takes an `enum_type` argument and registers a type with >> values >> >> taken from the enum value names. The actual values associated with the >> names >> >> are ignored, so support for IntEnum and other enum types comes as >> standard. >> >> >> >> In a real implementation, the enum type would have to be checked when >> >> running migrations to ensure that values haven't been added/removed >> from the >> >> python class. It's not something that we've needed to deal with in our >> >> in-house implementation. >> >> >> >> Any database which does not support an enum field natively would >> default >> >> to a CharField implementation. >> >> >> >> Useful addition? Or does it overlap too much with the choices API? >> >> >> >> Thomas >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> Groups >> >> "Django developers (Contributions to Django itself)" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> an >> >> email to django-develop...@googlegroups.com. >> >> To post to this group, send email to django-d...@googlegroups.com. >> >> Visit this group at http://groups.google.com/group/django-developers. >> >> To view this discussion on the web visit >> >> https://groups.google.com/d/msgid/django-developers/ >> 59072aa1-7e7a-4fcf-8dd1-effde66675c6%40googlegroups.com. >> >> For more options, visit https://groups.google.com/d/optout. >> > >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Django developers (Contributions to Django itself)" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to django-develop...@googlegroups.com. >> > To post to this group, send email to django-d...@googlegroups.com. >> > Visit this group at http://groups.google.com/group/django-developers. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/django-developers/CAMwjO1GAG88_% >> 3DLFRibpO6uabUmCb7eprByWRZyjECdV2jHbcxg%40mail.gmail.com. >> > >> > For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/2cd0cdcc-679e-4aef-929b-5129e3bc6f4c%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/2cd0cdcc-679e-4aef-929b-5129e3bc6f4c%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1HZQov%3DacCf%2Bbvbyxh1XJfSy3oWB7etBy9Mzo_MtTnzsA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
