Re: Massive import from CSV to Database

2011-11-16 Thread Andre Terra
Hey Fabio Sorry for not replying any sooner. It was a holiday here in Brazil, so I haven't been checking my e-mail that regularly. I promise to get back at you as soon as I can, okay? I'll be sure to provide some snippets so that you can get a sense of how I achieved this. Maybe I'll just make a

Re: Massive import from CSV to Database

2011-11-13 Thread Fabio Natali
On 11/12/2011 07:40 PM, Andre Terra wrote: Hello, Fabio I took a look at django-csv-importer but since my files weren't quite clean (or comma separated), I ended up rolling my own solution using celery[1], to make the uploading rask asynchronous, and DSE[2], to allow for faster bulk imports,

Re: Massive import from CSV to Database

2011-11-13 Thread Fabio Natali
On 11/13/2011 08:38 AM, Fabio Natali wrote: [...] Here is my try: ### from django.db import models from csvImporter.model import CsvModel class MyCsvModel(CsvModel): name = models.CharField() age = models.IntegerField() length = models.FloatField() class Meta: delimiter = ";" my_csv_list =

Re: Massive import from CSV to Database

2011-11-13 Thread Fabio Natali
On 11/12/2011 08:25 PM, Dan Poirier wrote: There are some ways to make large data imports much faster. See e.g. http://www.caktusgroup.com/blog/2011/09/20/bulk-inserts-django/ Hi Dan! Import speed was not my primary concern during these first attempts. I thought that django-csv-importer

Re: Massive import from CSV to Database

2011-11-12 Thread Fabio Natali
On 11/12/2011 07:40 PM, Andre Terra wrote: Hello, Fabio I took a look at django-csv-importer but since my files weren't quite clean (or comma separated), I ended up rolling my own solution using celery[1], to make the uploading rask asynchronous, and DSE[2], to allow for faster bulk imports,

Re: Massive import from CSV to Database

2011-11-12 Thread Dan Poirier
There are some ways to make large data imports much faster. See e.g. http://www.caktusgroup.com/blog/2011/09/20/bulk-inserts-django/ Dan -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to

Re: Massive import from CSV to Database

2011-11-12 Thread Andre Terra
Hello, Fabio I took a look at django-csv-importer but since my files weren't quite clean (or comma separated), I ended up rolling my own solution using celery[1], to make the uploading rask asynchronous, and DSE[2], to allow for faster bulk imports, as well as a lot of trial and error. Take a

Massive import from CSV to Database

2011-11-12 Thread Fabio Natali
Hi everybody! I am developing a Django website. I want the admin to be able to perform massive import of products into the database. This is the workflow: - the admin uploads a CSV file with a long list of products e.g.: product_1;230$;16kg;img_001.jpg;"A very good item of ours"