Re: Python, MS SQL, and batch inserts

2009-04-23 Thread Lawrence D'Oliveiro
In message mailman.4414.1240468019.11746.python-l...@python.org, Dennis Lee Bieber wrote: You don't mean to say you are creating full SQL insert statements WITH THE DATA! Nothing wrong with that http://codecodex.com/wiki/index.php?title=Useful_MySQL_Routines. --

Re: Python, MS SQL, and batch inserts

2009-04-23 Thread Lawrence D'Oliveiro
In message mailman.4293.1240337749.11746.python-l...@python.org, Philip Semanchuk wrote: Are you sure your logjam is in Python? Inserting 5500 rows can take a few seconds if you're COMMITting after each INSERT. Wrap the whole thing in an explicit transaction and see if that helps. Does

Re: Python, MS SQL, and batch inserts

2009-04-23 Thread Lawrence D'Oliveiro
In message mailman.4489.1240547176.11746.python-l...@python.org, Dennis Lee Bieber wrote: On Thu, 23 Apr 2009 21:00:48 +1200, Lawrence D'Oliveiro l...@geek-central.gen.new_zealand declaimed the following in gmane.comp.python.general: Nothing wrong with that

Re: Python, MS SQL, and batch inserts

2009-04-22 Thread pruebauno
On Apr 21, 5:21 pm, Scott David Daniels scott.dani...@acm.org wrote: ericwoodwo...@gmail.com wrote: On Apr 21, 4:01 pm, ericwoodwo...@gmail.com wrote: On Apr 21, 3:36 pm, Scott David Daniels scott.dani...@acm.org wrote: I forget the name of the SQL Server bulk loader, bcp (bulk

Python, MS SQL, and batch inserts

2009-04-21 Thread ericwoodworth
Hi, I have a python script I'm writing that grabs some data from a com object, does a little formatting, and then inserts that data into a MS SQL 2003 DB. Because I'm using COM objects I'm importing win32com.client. That also allows me to use ADODB.connection and ADODB.command objects for

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread Philip Semanchuk
On Apr 21, 2009, at 2:02 PM, ericwoodwo...@gmail.com wrote: Hi, I have a python script I'm writing that grabs some data from a com object, does a little formatting, and then inserts that data into a MS SQL 2003 DB. Because I'm using COM objects I'm importing win32com.client. That also

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread ericwoodworth
On Apr 21, 2:15 pm, Philip Semanchuk phi...@semanchuk.com wrote: On Apr 21, 2009, at 2:02 PM, ericwoodwo...@gmail.com wrote: Hi,     I have a python script I'm writing that grabs some data from a com object, does a little formatting, and then inserts that data into a MS SQL 2003 DB.  

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread Philip Semanchuk
On Apr 21, 2009, at 2:30 PM, ericwoodwo...@gmail.com wrote: I'm not 100% sure it's python and not SQL but I do suspect there's a better way to do this than just serial inserts. I could be wrong about that which is what i'm trying to explore. I don't think the SQL standard provides a way to

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread Scott David Daniels
Philip Semanchuk wrote: ... If you're doing a mass insert to populate a blank table it also often helps to postpone index creation until after the table is populated I forget the name of the SQL Server bulk loader, but for large loads, I used to populate a fresh table with the bulk data,

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread ericwoodworth
On Apr 21, 3:36 pm, Scott David Daniels scott.dani...@acm.org wrote: Philip Semanchuk wrote: ... If you're doing a mass insert to populate a blank table it also often helps to postpone index creation until after the table is populated I forget the name of the SQL Server bulk loader, but

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread ericwoodworth
On Apr 21, 4:01 pm, ericwoodwo...@gmail.com wrote: On Apr 21, 3:36 pm, Scott David Daniels scott.dani...@acm.org wrote: Philip Semanchuk wrote: ... If you're doing a mass insert to populate a blank table it also often helps to postpone index creation until after the table is

Re: Python, MS SQL, and batch inserts

2009-04-21 Thread Scott David Daniels
ericwoodwo...@gmail.com wrote: On Apr 21, 4:01 pm, ericwoodwo...@gmail.com wrote: On Apr 21, 3:36 pm, Scott David Daniels scott.dani...@acm.org wrote: I forget the name of the SQL Server bulk loader, bcp (bulk copy) was the name of the bulk loader. I just remembered. Sorry, no