urllib2 script slowing and stopping

2010-10-11 Thread Dantium
I have a small script that reads several CSV files in a directory and puts the data in a DB using Django. There are about 1.7 million records in 120 CSV files, I am running the script on a VPS with about 512mb of memory python 2.6.5 on ubuntu 10.04. The script gets slow and seems to lock after

Re: urllib2 script slowing and stopping

2010-10-11 Thread Dantium
On Oct 11, 10:07 pm, Ian ian.g.ke...@gmail.com wrote: On Oct 11, 2:48 pm, Dantium dan...@gmail.com wrote: I have a small script that reads several CSV files in a directory and puts the data in a DB using Django. There are about 1.7 million records in 120 CSV files, I am running