+1 for this feature addition. I work with a lot of public data and I almost always go through the steps of loading data into PostgreSQL and building from there. COPY reduces data load time significantly so a core load command for Django is welcome.
I've used LOAD DATA INFILE for MySQL on occasion too, but I'd rather stick to Django's ORM (and PostgreSQL) instead of writing raw SQL as Ben mentioned. Another neat idea is to borrow from GeoDjango's ogrinspect <https://docs.djangoproject.com/en/1.8/ref/contrib/gis/commands/#django-admin-ogrinspect> and do the same thing for CSVs (I wrote a simple implementation here <https://github.com/cirlabs/django-project-template/blob/master/lib/management/commands/csv_to_model.py>). These two tools together will be huge leap in productivity for the data journalism / open data communities. I should add I just used django-postgres-copy for a project I'm working on and it ran like a charm. On Saturday, July 18, 2015 at 3:19:49 PM UTC-7, Ben Welsh wrote: > > Hello, > > I am a big fan of PostgreSQL's excellent bulk loader COPY > <http://www.postgresql.org/docs/9.4/static/sql-copy.html>. It can rapidly > load a CSV file directly into the database, an oftentimes tedious task that > I have to do frequently as part of my job. > > I am not a big fan of having to write my COPY commands in raw SQL. I'd > much rather use Django's ORM. > > So last week I put together an app call django-postgres-copy > <http://django-postgres-copy.californiacivicdata.org/en/latest/> that > attempts to integrate COPY into Django, modeling its design on the > excellent LayerMapping > <https://docs.djangoproject.com/en/1.8/ref/contrib/gis/layermapping/> > utility in contrib.gis, which I also use frequently. > > I wrote a blog post about the approach here > > http://www.californiacivicdata.org/2015/07/17/hello-django-postgres-copy/ > > You can find more complete technical documentation here > > http://django-postgres-copy.californiacivicdata.org/en/latest/ > > And all of the code is up here on GitHub > > https://github.com/california-civic-data-coalition/django-postgres-copy > > Since Django has already begun to integrate other PostgreSQL-specific > features in contrib.postgres, I'm curious if the core developers are be > interested in adding COPY support as well. > > I'm not attached to the style of my library and I'd welcome a different > approach if it got the job done. I'd be thrilled to have the opportunity to > carry the torch and do whatever refactoring and additional coding is > necessary to qualify it for a merge. > > Please let me know what you think. And if I've overlooked some previous > discussion or superior third-party library in this area, please forgive me. > I searched around but was unable to find anything. > > Sincerely, > > Ben Welsh > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a128fbdb-181f-400b-b810-8c451f735a62%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
