heya,
Hmm, I was previously under the impression that for these sorts of things
(importing and instantiating models from CSV), the recommended way to
create a ModelForm, and pass each line of the CSV through that, and have
that handle model validation for you.
In our case, we have a CSV file,
On Sun, Dec 4, 2011 at 2:53 AM, Karen Tracey wrote:
> On Sat, Dec 3, 2011 at 9:37 PM, Nathan McCorkle wrote:
>>
>> when you say 'as long as you don't use the ORM for inserts',
>
>
> You do not want to be building and saving objects individually with the ORM.
> You want to be using some form of bu
I think it's been mentioned before in this thread, but DSE was made to
help solve problems this. The other solutions I've seen for this
problem, like the one mentioned by Karen, lack several of DSE's
features, like handling of default values defined in your model.
https://bitbucket.org/weholt/dse2
On Sat, Dec 3, 2011 at 9:37 PM, Nathan McCorkle wrote:
> when you say 'as long as you don't use the ORM for inserts',
You do not want to be building and saving objects individually with the
ORM. You want to be using some form of bulk insert. Django 1.4 will add
bulk create capability (see
https
On Fri, Dec 2, 2011 at 6:22 AM, Cal Leeming [Simplicity Media Ltd]
wrote:
> Faster in what sense? Prototyping/development time, or run time?
Well I can count the lines in each file in a few seconds, so I think
the SQL stuff is slowing everything down (using postgres through
psycodb2)
>
> If it's
Faster in what sense? Prototyping/development time, or run time?
If it's only a few MB, I see little reason to go as far as to writing it in
C. Unless you are performing the same import tens of thousands of times,
and the overhead in Python adds up so much that you get problems.
But, quite frankl
would interfacing with SQL via C or C++ be faster to parse and load
data in bulk? I have files that are only a few MB worth of text, but
can take hours to load due to the amount of parsing I do, and the
number of database entries each item in a file makes
On Mon, Nov 28, 2011 at 3:28 AM, Anler Her
On 11/28/2011 09:28 AM, Anler Hernandez Peral wrote:
Hi, this is probably not your case, but in case it is, here is my story:
Creating a script for import CSV files is the best solution as long as
they are few, but in my case, the problem was that I need to import
nearly 40 VERY BIG CSV files, ea
On 11/27/2011 07:56 PM, Andre Terra wrote:
This should be run asynchronously (i.e. celery) when importing large files.
If you have a lot of categories/subcategories, you will need to bulk
insert them instead of looping through the data and just using
get_or_create. A single, long transaction wil
On 11/26/2011 11:44 PM, Petr Přikryl wrote:
import csv
data = csv.reader(open('/path/to/csv', 'r'), delimiter=';')
for row in data:
category = Category.objects.get_or_create(name=row[0])
sub_category = SubCategory.objects.get_or_create(name=row[1],
defaults={'parent_category': category})
produc
Hi, this is probably not your case, but in case it is, here is my story:
Creating a script for import CSV files is the best solution as long as they
are few, but in my case, the problem was that I need to import nearly 40
VERY BIG CSV files, each one mapping a database table, and I needed to do
it
This should be run asynchronously (i.e. celery) when importing large files.
If you have a lot of categories/subcategories, you will need to bulk insert
them instead of looping through the data and just using get_or_create. A
single, long transaction will definitely bring great improvements to spee
>>> import csv
>>> data = csv.reader(open('/path/to/csv', 'r'), delimiter=';')
>>> for row in data:
>>> category = Category.objects.get_or_create(name=row[0])
>>> sub_category = SubCategory.objects.get_or_create(name=row[1],
>>> defaults={'parent_category': category})
>>> product = Product.objects
On 11/25/2011 05:23 PM, Fabio Natali wrote:
On 11/25/2011 03:12 PM, Tom Evans wrote:
[...]
It's not that tricky, is it?
Read the CSV file, split out the fields.
Get or create the category
Get or create the subcategory
Get or create the product
in code:
import csv
data = csv.reader(open('/path
On 11/25/2011 03:12 PM, Tom Evans wrote:
[...]
It's not that tricky, is it?
Read the CSV file, split out the fields.
Get or create the category
Get or create the subcategory
Get or create the product
in code:
import csv
data = csv.reader(open('/path/to/csv', 'r'), delimiter=';')
for row in dat
On Fri, Nov 25, 2011 at 2:03 PM, Fabio Natali wrote:
> Hi everybody!
>
> I have a CSV file and have to import it in my Django website.
>
> Say I have three models involved: category, sub_category and product.
>
> ### models.py ###
> class Category(models.Model):
> name = models.CharField(max_le
Hi everybody!
I have a CSV file and have to import it in my Django website.
Say I have three models involved: category, sub_category and product.
### models.py ###
class Category(models.Model):
name = models.CharField(max_lenth=100)
class SubCategory(models.Model):
name = models.CharFi
17 matches
Mail list logo
