Re: [sqlalchemy] Speed up bulk inserts

2014-08-19 Thread Michael Bayer
sure that feature is here: http://docs.sqlalchemy.org/en/rel_0_9/core/dml.html?highlight=insert%20values#sqlalchemy.sql.expression.Insert.values.params.*args On Aug 18, 2014, at 7:32 PM, Anton anschat...@gmail.com wrote: Hi Michael, I checked your post on the stackoverflow, could you help

Re: [sqlalchemy] Speed up bulk inserts

2014-08-18 Thread Anton
Hi Michael, I checked your post on the stackoverflow, could you help me figure something out regarding usage of Core. In the example you provided you have a version with test_sqlite3 which uses customer user supplied query so it issues only one insert statement with all rows in values clause.

Re: [sqlalchemy] Speed up bulk inserts

2013-11-13 Thread Achim
Am Mittwoch, 6. November 2013 21:58:53 UTC+1 schrieb Michael Bayer: I wrote a full post regarding this topic on stackoverflow at http://stackoverflow.com/questions/11769366/why-is-sqlalchemy-insert-with-sqlite-25-times-slower-than-using-sqlite3-directly/11769768#11769768. If you start

Re: [sqlalchemy] Speed up bulk inserts

2013-11-13 Thread Michael Bayer
On Nov 13, 2013, at 4:57 AM, Achim do...@procoders.net wrote: Am Mittwoch, 6. November 2013 21:58:53 UTC+1 schrieb Michael Bayer: I wrote a full post regarding this topic on stackoverflow at

Re: [sqlalchemy] Speed up bulk inserts

2013-11-13 Thread Michael Bayer
On Nov 13, 2013, at 11:52 AM, Claudio Freire klaussfre...@gmail.com wrote: On Wed, Nov 13, 2013 at 1:45 PM, Michael Bayer mike...@zzzcomputing.com wrote: Am Mittwoch, 6. November 2013 21:58:53 UTC+1 schrieb Michael Bayer: I wrote a full post regarding this topic on stackoverflow at

Re: [sqlalchemy] Speed up bulk inserts

2013-11-13 Thread Achim
According to http://www.postgresql.org/docs/9.2/static/functions-sequence.html, sequences are non-transactional: Advance the sequence object to its next value and return that value. This is done atomically: even if multiple sessions execute nextval concurrently, each will safely receive a

Re: [sqlalchemy] Speed up bulk inserts

2013-11-13 Thread Claudio Freire
On Wed, Nov 13, 2013 at 2:04 PM, Michael Bayer mike...@zzzcomputing.com wrote: On Nov 13, 2013, at 11:52 AM, Claudio Freire klaussfre...@gmail.com wrote: On Wed, Nov 13, 2013 at 1:45 PM, Michael Bayer mike...@zzzcomputing.com wrote: Am Mittwoch, 6. November 2013 21:58:53 UTC+1 schrieb

Re: [sqlalchemy] Speed up bulk inserts

2013-11-13 Thread Michael Bayer
On Nov 13, 2013, at 12:22 PM, Claudio Freire klaussfre...@gmail.com wrote: On Wed, Nov 13, 2013 at 2:04 PM, Michael Bayer mike...@zzzcomputing.com wrote: On Nov 13, 2013, at 11:52 AM, Claudio Freire klaussfre...@gmail.com wrote: On Wed, Nov 13, 2013 at 1:45 PM, Michael Bayer

[sqlalchemy] Speed up bulk inserts

2013-11-06 Thread Achim Domma
Hi, I want to speed up my SqlAlchemy bulk inserting code and yes, I'm aware that this is not the main purpose of SqlAlchemy and all databases have faster low level import tools. The background is the following: We import data from various sources and apply various mappings. Currently I'm

Re: [sqlalchemy] Speed up bulk inserts

2013-11-06 Thread Michael Bayer
I wrote a full post regarding this topic on stackoverflow at http://stackoverflow.com/questions/11769366/why-is-sqlalchemy-insert-with-sqlite-25-times-slower-than-using-sqlite3-directly/11769768#11769768 . If you start with this, I can answer more specific questions. On Nov 6, 2013, at 10:28