+1 to mike's suggestion of batches to weed out potential dupes. that will save a lot.
you'd also have to do some pre-processing within your 1000, to make sure one member of that set won't violate a constraint created by another. if your DB supports savepoints, you can also add in a savepoint/nested transaction and then immediately flush. you'll be able to avoid most errors like that. i use postgres, and I found the best option FOR ME was to use Python+SqlAlchemy to query and track the duplicates -- and then to generate a text file in the postgres COPY FROM format for bulk import. I don't know if other dbs have a similar function. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
