Adding a database-agnostic JSONField into Django
Hello, in almost all projects I work on, I end up using a JSONField. Since I value being able to run with any database, I'm not relying on django.contrib.postgres.fields.JSONField. So I have been using pypi's django-jsonfield maintained by Matthew Schinckel: https://bitbucket.org/schinckel/django-jsonfield (I have also packaged this for Debian) I have recently discovered pypi's "jsonfield" maintained by Brad Jasper: https://github.com/bradjasper/django-jsonfield Both projects are very similar (and use the same python package name) and both projects are actually looking for a new maintainer... since I rely on something like this, I would be willing to try to merge the best of both modules into a possible django.contrib.jsonfield or directly into the core. We could use this opportunity to let the newly-integrated field use DjangoJSONEncoder by default (see recent discussion about this) and django.contrib.postgres could register its additionals lookups into the generic field (assuming we use "jsonb" as underlying type for postgresql). What do you think of this? If inclusion into Django is not desired, then maybe we could aim to at least merge both of those projects in a single "blessed" third-party module that could be maintained in https://github.com/django/django-jsonfield? Cheers, -- Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20160623095656.GB22598%40home.ouaza.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding a database-agnostic JSONField into Django
I would also love to see this in Django. Although I mostly use Postgres, it would be nice not to be locked into a single database vendor if I were to use a JSONField. On 23 Jun 2016 7:57 PM, "Raphael Hertzog" wrote: > Hello, > > in almost all projects I work on, I end up using a JSONField. Since > I value being able to run with any database, I'm not relying on > django.contrib.postgres.fields.JSONField. So I have been using > pypi's django-jsonfield maintained by Matthew Schinckel: > https://bitbucket.org/schinckel/django-jsonfield > (I have also packaged this for Debian) > > I have recently discovered pypi's "jsonfield" maintained by Brad Jasper: > https://github.com/bradjasper/django-jsonfield > > Both projects are very similar (and use the same python package name) and > both projects are actually looking for a new maintainer... since I rely on > something like this, I would be willing to try to merge the best of both > modules into a possible django.contrib.jsonfield or directly into the core. > > We could use this opportunity to let the newly-integrated field use > DjangoJSONEncoder by default (see recent discussion about this) and > django.contrib.postgres could register its additionals lookups into the > generic field (assuming we use "jsonb" as underlying type for postgresql). > > What do you think of this? > > If inclusion into Django is not desired, then maybe we could aim to > at least merge both of those projects in a single "blessed" third-party > module that could be maintained in > https://github.com/django/django-jsonfield? > > Cheers, > -- > Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer > > Discover the Debian Administrator's Handbook: > → http://debian-handbook.info/get/ > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/20160623095656.GB22598%40home.ouaza.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAKL8yL63BqvXL_MO-F9NmsnMJ%3DZq58WkdvUPCJGJFwv%2Br61m%3Dg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Fate of sql* management commands
On Thursday, June 23, 2016 at 12:12:51 AM UTC+2, Shai Berger wrote: > > > Thanks for this thoughtful clarification. I think I understand your > position > much better now. If Thanks for reading, Shai. I know my English is far from perfect, so I appreciate your involvement. I understand correctly, there are two issues you find with > migrations: > > - They are designed to deal with schema changes, while you'd rather have a > tool for one-time schema generation; > > In most cases, yes. But I see now new possibilities (I'll wrote about my thoughts another time) > - Migration files are Python code with potential dependencies, while you > prefer > your schema (and even data) changes to be expressed in pure SQL. > > This is just one of all reasons, but yes. > > It was very handy to prototype the app layer and generate DDL. Then DDL > was > > used directly in db schema management system. > > - Isn't it, then, possible to generate a schema by evolving it while > prototyping on your development machine, and when done, use the db schema > management tools to extract the DDL directly from the database? Isn't it > actually better than generating DDL in Python, according to your views? > > It depends on what "better" means. Django is very helpful for generating initial sqls, especially for indexes and foreign keys. Further changes we make using db mgmt tool. This is part of our workflow for years - prototype python code, make some tests, generate sql for new models, make additional sqls (views, triggers, etc), finish feature, send to qa, deliver. Writing Django models in Python is faster than writing plain SQL or using db GUI tools. And we are writing them just only once. Using other DDL tools and making diffs between databases will slow down our workflow. We're accepting Django's table/columns/m2m/constraints naming conventions, and we're accepting all limitiations. The db schema is compatible with Django. We didn't found faster method yet (except Django's changes autodetection). > - But Andrew's patch[1] completely ignores any existing migration files, > bypassing all problems related to migration files as well as the "journey > through winding road". If it will not use migration files when they already exists, it sounds ok. I know too little about migations internals, so I can't be sure for now. > What he suggests is to use the migrations > infrastructure to generate just the SQL for the difference from "nothing" > to > "current models" (so, no "model state changes" either). I thought that diff from nothing to current will be achieved by applying migrations sequentially. It sound ok if this will be ommitted. > Even the issue of > SchemaEditor's deferred_sql is handled by "collect_sql" mode, as you've > already noted yourself (and by the way, the deferring mechanism is > required in > order to resolve circular dependencies between models). Do you have other > objections to this patch's approach? > Well.. yes and no. I just have almost finished patch which brings back sql* commads based directly on SchemaEditor. The difference is in ommitting django.db.migrations completely, so there is straightforward path to get current db state (without using whole complex migrations abstraction layer), less dependencies and less risk of failure due to some db.migrations issues/regression. The drawback is that output sql may little differ comparing to output from migrations-based alternative, even if both methods will use same SchemaEditor. There also will be new association to SE, so further changes of SE will require attention not only to migrations subsystem. Please share your thougths because I can miss something. I don't how important is to use django.db.migrations for every sql-related use case. It is about your preferences and design decisions. For me it is not a problem until both (sql* and migrations) depends on same sql factory (SchemaEditor here). For me, the sql* world is quite different from the migrations, and it is ok to not mix up these two things. The cost of maintaining additional association seems to be low, but I might be wrong. The "dead end" term used by me in the context of Andrew's patch was mostly related to the migration files issues. Having already better knoweldge I will look at his pull rq again. Marcin -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2c4cbc51-d9e3-401e-a8ab-fdb3e35d9c9a%40googlegroups.com. For more options, visit https://groups.google.c
Re: Adding a database-agnostic JSONField into Django
Same boat for me. I constantly need the JSON field but can't always rely on postgres. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAGuvt930QvCKazawJbH_OpHwqSpSw74TtgEKR1Tpp2Yh23WLuA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding a database-agnostic JSONField into Django
1 up on this one, would be really useful for me as well! On Thursday, 23 June 2016 10:57:07 UTC+1, Raphael Hertzog wrote: > > Hello, > > in almost all projects I work on, I end up using a JSONField. Since > I value being able to run with any database, I'm not relying on > django.contrib.postgres.fields.JSONField. So I have been using > pypi's django-jsonfield maintained by Matthew Schinckel: > https://bitbucket.org/schinckel/django-jsonfield > (I have also packaged this for Debian) > > I have recently discovered pypi's "jsonfield" maintained by Brad Jasper: > https://github.com/bradjasper/django-jsonfield > > Both projects are very similar (and use the same python package name) and > both projects are actually looking for a new maintainer... since I rely on > something like this, I would be willing to try to merge the best of both > modules into a possible django.contrib.jsonfield or directly into the > core. > > We could use this opportunity to let the newly-integrated field use > DjangoJSONEncoder by default (see recent discussion about this) and > django.contrib.postgres could register its additionals lookups into the > generic field (assuming we use "jsonb" as underlying type for postgresql). > > What do you think of this? > > If inclusion into Django is not desired, then maybe we could aim to > at least merge both of those projects in a single "blessed" third-party > module that could be maintained in > https://github.com/django/django-jsonfield? > > Cheers, > -- > Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer > > Discover the Debian Administrator's Handbook: > → http://debian-handbook.info/get/ > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/517f7674-b5fe-4239-9df1-e1f94d135763%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Adding a database-agnostic JSONField into Django
Making JSONField a core field was discussed in https://groups.google.com/d/topic/django-developers/sAgYOqBUvgI/discussion Adam said, "my recent completion of its *JSONField* for MySQL 5.7+ is very similar to the *contrib.postgres* one, copying and adapting large parts of code from Marc Tamlyn's work. We all know how much everyone loves JSON these days. If anything, this could be a core field rather than a *contrib* one - Oracle and SQLite also have JSON capabilities now. JSON everywhere!" Marc said, "I'm happy for JSONField to be made a core field on the condition that it's underlying support is more than a text blob on all our main databases. It sounds like this will soon be the case." On Thursday, June 23, 2016 at 8:43:33 AM UTC-4, tedjohanss...@gmail.com wrote: > > 1 up on this one, would be really useful for me as well! > > On Thursday, 23 June 2016 10:57:07 UTC+1, Raphael Hertzog wrote: >> >> Hello, >> >> in almost all projects I work on, I end up using a JSONField. Since >> I value being able to run with any database, I'm not relying on >> django.contrib.postgres.fields.JSONField. So I have been using >> pypi's django-jsonfield maintained by Matthew Schinckel: >> https://bitbucket.org/schinckel/django-jsonfield >> (I have also packaged this for Debian) >> >> I have recently discovered pypi's "jsonfield" maintained by Brad Jasper: >> https://github.com/bradjasper/django-jsonfield >> >> Both projects are very similar (and use the same python package name) and >> both projects are actually looking for a new maintainer... since I rely >> on >> something like this, I would be willing to try to merge the best of both >> modules into a possible django.contrib.jsonfield or directly into the >> core. >> >> We could use this opportunity to let the newly-integrated field use >> DjangoJSONEncoder by default (see recent discussion about this) and >> django.contrib.postgres could register its additionals lookups into the >> generic field (assuming we use "jsonb" as underlying type for >> postgresql). >> >> What do you think of this? >> >> If inclusion into Django is not desired, then maybe we could aim to >> at least merge both of those projects in a single "blessed" third-party >> module that could be maintained in >> https://github.com/django/django-jsonfield? >> >> Cheers, >> -- >> Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer >> >> Discover the Debian Administrator's Handbook: >> → http://debian-handbook.info/get/ >> > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/889b5dba-d657-4b34-a878-028033956901%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Fate of sql* management commands
Hi Marcin, Thanks for taking the time to clarify constructively your use case. On 23 Jun 2016, at 13:10, Marcin Nowak wrote: > If it will not use migration files when they already exists, it sounds ok. I believe that this point was a big misunderstanding in the discussion. As far as I understand, Andrew’s proposal works pretty much like Django’s former SQL generation tools, except it builds upon the robust abstractions that support the migrations system — specifically the schema editor class, which is really better at generating SQL that the pre-1.7 implementation. Since migrations are designed to perform schema changes, we have to ask "create the SQL to turn into " instead of "create the SQL to generate ". The result is the same, however: Django will introspect the current state of models, build an in-memory representation of them, and generate the corresponding SQL. This doesn’t require writing out migration files to disk. So, the only outstanding issue is to complete the patch that implements this feature :-) Best regards, -- Aymeric. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4463F61A-8DDD-4F08-B0A9-23060768DF85%40polytechnique.org. For more options, visit https://groups.google.com/d/optout.
Re: Adding a database-agnostic JSONField into Django
Hello, On 23 Jun 2016, at 15:40, Tim Graham wrote: > Marc said, "I'm happy for JSONField to be made a core field on the condition > that it's underlying support is more than a text blob on all our main > databases. It sounds like this will soon be the case.” In order to discourage people from using JSONField on databases where it cannot be implemented efficiently i.e. not with a text blob, the best solution may be to add it to Django and raise a warning when it’s misused. I’m not sure we can do that with a check, though, as we can’t introspect database routing logic (without adding a disproportionate amount of complexity). It would have to be a runtime warning. -- Aymeric. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/45280562-6205-415F-B603-60837922D13F%40polytechnique.org. For more options, visit https://groups.google.com/d/optout.
Django Model Fields a repr_output property, to include a field in the string representation of an object
Has a feature like this ever been considered? If a model has no __unicode__, __str__ or __repr__ representation, then maybe it could devise a string representation by collecting fields which have this value set to True. Example: Without the feature: class Person(models.Model): first_name = models.CharField(max_length=50) last_name = models.CharField(max_length=50) >>> person = Person(first_name='Ben', last_name='Friedland') >>> print person # fairly useless object representation This feature would work something like: class Person(models.Model): first_name = models.CharField(max_length=50, *repr_output=True*) last_name = models.CharField(max_length=50, *repr_output=True*) >>> person = Person(first_name='Ben', last_name='Friedland') >>> print person # includes fields specified via repr_output=True If this would be useful I'd be happy to formally create an issue and even implement the feature. Thanks! Ben Friedland www.bugben.com -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ec667640-b82b-4d9e-a87a-b2dc705b5aa6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.