[sqlalchemy] Re: large amount of insert
Michael Bayer ha scritto: well at the very least you want to ensure that executemany() is being used for the inserts (i.e. with sqlalchemy, dont use ORM, use connection.execute(statement, [param, param, param...]). If you use SQLAlchemy's executemany() facilities, we've just increased their efficiency by about 60% in the 0.4.0beta3 release, we had some huge latency issues with this particular operation previously. faster still is to use raw DBAPI: conn = engine.connect() conn.connection.executemany(statement, [params...]) If this don't cause thread or other problem with SA, i think this is the best solution, i can split my n insert query in segments of 100 qry a time. using copy from..file is probably very fast, but its a little weird to use that in the context of a web request. I think too... Thank you -- ++ Glauco Uri - Programmatore glauco(at)allevatori.com Sfera Carta Software(r) [EMAIL PROTECTED] Via Bazzanese,69 Casalecchio di Reno(BO) - Tel. 051591054 ++ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: large amount of insert
there was some recent thread on this 2-3 weeks ago, lookup.. On Friday 17 August 2007 11:28:34 Glauco wrote: What's is the best solution for a web procedure , in TurboGear, that produce a large amount of insert into ? (from 2000 to 2 insertions on a submit) i've done some try with somethink like 5000 insertion and the transaction is huge do you think sqlalchemy can do this better with some trick or is better to transfer all data in a copy from file? Thank you Glauco --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: large amount of insert
well at the very least you want to ensure that executemany() is being used for the inserts (i.e. with sqlalchemy, dont use ORM, use connection.execute(statement, [param, param, param...]). If you use SQLAlchemy's executemany() facilities, we've just increased their efficiency by about 60% in the 0.4.0beta3 release, we had some huge latency issues with this particular operation previously. faster still is to use raw DBAPI: conn = engine.connect() conn.connection.executemany(statement, [params...]) using copy from..file is probably very fast, but its a little weird to use that in the context of a web request. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
