I've been trying to speed up a bulk insert process that currently seems too slow. I've read the past threads about how to replace the orm/session based inserts with table.insert().execute(*valuedicts) but in my case the objects are related (via 1-to-many relations if it makes a difference). In order to insert a child instance, the parent instance(s) has/ve to be inserted first so that the auto-generated ids can be assigned as foreign keys to the child. I guess that's what session.commit() does behind the scene anyway, so does it make sense to replicate this complexity (e.g. the topological sorting implied by the child-parent relationships) outside the orm ? Is there a faster way ?
Or maybe it's already fast enough and I have unrealistic expectations ? Currently it inserts ~300K total rows (spread across 3 tables - 25K / 110K / 165K) in little less than half an hour, ~170 rows/second. That's in a PostgreSQL database running with synchronous_commit turned off and all foreign and unique constraints dropped during the bulk load. George --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
