Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-09 Thread Jonathan Vanasco
On Wednesday, March 9, 2016 at 3:17:26 PM UTC-5, jbrow...@clearcosthealth.com wrote: > > Like I said I am not terribly network savvy but what I meant was that the > particular vpn client that I am using says that my "connection speed" is > ~540Mpbs. > That's most-likely a speed to your router

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-09 Thread jbrownstein
Like I said I am not terribly network savvy but what I meant was that the particular vpn client that I am using says that my "connection speed" is ~540Mpbs. I know comparing sql alchemy to raw sql is kind of a no-no but I was just illustrating that the there isn't obvious evidence of a slow net

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-09 Thread Jonathan Vanasco
Those aren't the same inserts as SqlAlchemy The SQL you noted above (DECLARE / BEGIN/ END) is one packet of text that is sent over the wire to the database server and executed. (a few hundred bytes) A more appropriate comparison would be sending 1000 separate "INSERT INTO " statements ove

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-09 Thread Mike Bayer
On 03/09/2016 11:56 AM, jbrownst...@clearcosthealth.com wrote: This is all extremely helpful. After some hair pulling I was able to get the local sql server express running and I ran the same test on a local instance. 10100.1260.0000.1260.000 {method 'execute' of 'pyodbc

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-09 Thread jbrownstein
This is all extremely helpful. After some hair pulling I was able to get the local sql server express running and I ran the same test on a local instance. 10100.1260.0000.1260.000 {method 'execute' of 'pyodbc.Cursor' objects} 10100.0600.0000.0600.000 {m

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-08 Thread Mike Bayer
so the profile shows pretty clearly that the time is spent on the DB side. 1006 SQL statements are clocking at .09 seconds each (!) to take a total of 93 seconds: 10060.0120.000 93.8730.093 C:\Users\jbrownstein\Documents\GitHub\ETL\ETL\env\lib\site-packages\sqlalchemy\engine\b

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-08 Thread jbrownstein
I'm on sqla 1.0.12 / MS SQL Server 2014 / python 64bit 2.7 (sorry should have included that in the first post) Here is an extract from the profile using the code from http://docs.sqlalchemy.org/en/rel_0_8/faq.html#code-profiling I am reading explanations in the FAQ now but any guidance would be

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-07 Thread Jason Brownstein
Thanks Mike - will the full profile ASAP. On Monday, March 7, 2016, Mike Bayer wrote: > What will make inserts slow is network as well as indexes. If you aren't > comparing your script to some other backend then there's probably a whole > lot of other things that can get in the way like unicod

Re: [sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-07 Thread Mike Bayer
What will make inserts slow is network as well as indexes. If you aren't comparing your script to some other backend then there's probably a whole lot of other things that can get in the way like unicode encoding, large columns or other surprises in the model. If you run the full profile ver

[sqlalchemy] session flush inserts performing VERY slow for MS Sql Server

2016-03-07 Thread jbrownstein
I've run sqlalchemy on MySQL dbs and never had a problem with performance, but when I started writing an app that connected to Microsoft SQL Server 2014 via pyodbc I ran into a serious performance problem. I'm running the app on my laptop and the database server IS in a VPN but my internet is f

Re: [sqlalchemy] Session flush trigger misunderstanding or unexpected behaviour

2016-03-04 Thread Bastien Sevajol
Hi Simon King, Thank's for your reply. autoflush disabling effectivly help me. About my functional problem, i finally use an other workflow including context manager usage. Regards, Bastien. Le mardi 1 mars 2016 12:28:51 UTC+1, Simon King a écrit : > > > The answer to your first question is th

Re: [sqlalchemy] Session flush trigger misunderstanding or unexpected behaviour

2016-03-01 Thread Simon King
On Tue, Mar 1, 2016 at 9:43 AM, Bastien Sevajol wrote: > Hi, > > I'm confronted to an misunderstanding or unexpected behavior of session > trigger flush. I want to manage revision of a document, like in this >

[sqlalchemy] Session flush trigger misunderstanding or unexpected behaviour

2016-03-01 Thread Bastien Sevajol
Hi, I'm confronted to an misunderstanding or unexpected behavior of session trigger flush. I want to manage revision of a document, like in this doc example. But the behavior is unexpected: I want to do

[sqlalchemy] session flush

2009-06-17 Thread Michael Mileusnich
I know I have asked this before but I would like some clarification. If I do something like : i = query(table).all for instance in i: thread_class(i.ID) thread_class.start() del thread_class and do an additional query in the init of the thread_class to pull up the entire record for i using the