[sqlalchemy] Inserting Entry Fastest way
Hi, I am working on a database design that I am required to use lots of tables with one-to-many relationship. As a consequence of the design, I need to insert thousands of entries. I tried session.add(), session.merge, however none of them is fast enough for me to meet the requirements. I was wondering if you can suggest me an efficient way through either ORM or ORM+Core. Thanks! -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Inserting Entry Fastest way
I wrote a detailed description of what's going on regarding INSERT here: http://stackoverflow.com/questions/11769366/why-is-sqlalchemy-insert-with-sqlite-25-times-slower-than-using-sqlite3-directly/11769768#11769768 On Mar 11, 2013, at 4:56 PM, Arkilic, Arman arki...@bnl.gov wrote: Hi, I am working on a database design that I am required to use lots of tables with one-to-many relationship. As a consequence of the design, I need to insert thousands of entries. I tried session.add(), session.merge, however none of them is fast enough for me to meet the requirements. I was wondering if you can suggest me an efficient way through either ORM or ORM+Core. Thanks! -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Inserting Entry Fastest way
On Mar 11, 2013, at 5:19 PM, Russ russandheat...@gmail.com wrote: On Monday, March 11, 2013 4:56:11 PM UTC-4, Arkilic, Arman wrote: Hi, I am working on a database design that I am required to use lots of tables with one-to-many relationship. As a consequence of the design, I need to insert thousands of entries. I tried session.add(), session.merge, however none of them is fast enough for me to meet the requirements. I was wondering if you can suggest me an efficient way through either ORM or ORM+Core. Thanks! I recently did a presentation structured around getting fast bulk inserts with SQLAlchemy. You may find it useful: https://speakerdeck.com/rwarren/a-brief-intro-to-profiling-in-python Please note that the focus was on profiling, and not on SQLAlchemy. The SQLAlchemy example just worked out well (with a contrived step or two) as a vehicle for showing different profiling steps/gotchas. Since the focus was on profiling, the example is quite simple (a single user table)... but you can easily extend on it for your one-to-many tables. I also didn't 100% scrub the SQLAlchemy code (I threw this together in a hurry), so no yelling at me for bad code. :) wow that is a great talk, I laughed my ass off and you really got in there, nice job ! Russ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Inserting Entry Fastest way
wow that is a great talk, I laughed my ass off and you really got in there, nice job ! Thanks! As long as you weren't laughing because I did the sqlalchemy all wrong! :) Russ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
