On Tuesday, December 30, 2014 9:54:10 PM UTC-8, Martin Mirero wrote: > > Hi folks, > > I'm just cutting my teeth on Python/Django and need some assistance on > something I've been grappling with for a few days: > > - I have a bunch of text files on disk that all have the same basic > format: first line is the *title* and the rest is the *body* > - I want to create one CSV file with the first column being populated > from the first line of each text file (title) and the second column being > populated from the rest of the text file contents (body) > - Once I have the CSV file, I'm good to import that into my django > model > > Depending on your use case, there may be an easier (or more efficient way) to skin this cat. Both mysql and postgres have syntax that will let you load an entire csv into a db table as a one-liner. To do that from within Django, you need to be able to run an external shell command, and the best way to do that is (often) with Fabric. And since this is a function you'll likely want to do on a regular basis, it makes sense to wrap it all up in a Django management command.
Here's an excerpt of some code I use to do something similar: http://pastebin.com/crpnNVrx This is located in someapp/management/commands/import_courses.py . With this in place I can run ./manage.py import_courses at any time. On each run, it: - Drops a temporary table - Re-creates that temporary table according to a predefined postgres schema (contained in importer_courses.sql) - Imports the CSV into that temporary table In my case, I needed to be able to perform queries across the CSV data *before* importing some of it into my real models, but this turned out to be a wonderfully efficient way of handling CSV data in general. You don't have to drop and recreate the temp table each time - I chose to do it that way for my use case but season to taste. To use this technique you'll need to `pip install fabric`, set a directory location where your CSVs are located (as IMPORTER_DATA_DIR in your settings), and read up a bit on Fabric and Django management commands. Then you'll need to modify it to handle multiple CSVs rather than just one. ./s -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/178e2096-a9b7-4f92-8d7a-f375371a527e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
