Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-13 Thread Michael Bayer
OK well of course also, as we have the exact same thing being asked in regards to Oracle right now in another thread, you can of course always bypass a bound value in the most direct way, using text() or literal_column(): q = s.query(Something).filter(Something.foo = literal_column('my value'))

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-13 Thread Seth P
Yep, literal_column() fixed my performance problem. Thanks again for all your help On Tuesday, May 13, 2014 6:15:39 AM UTC-4, Michael Bayer wrote: OK well of course also, as we have the exact same thing being asked in regards to Oracle right now in another thread, you can of course always

[sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Seth P
After tracking down some extreme slowness in loading a one-to-many relationship (e.g. myobject.foobars), I seem to have isolated the issue to engine.execute() being much slower with parameterized queries than with explicit queries. The following is actual code and output for loading 10,971

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Michael Bayer
well there's only one parameter being processed here so there is clearly negligible difference in time spent within Python as far as getting the statement ready to execute and then executing it. So the time is either in what SQL Server spends fetching the rows, or the number of rows being

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Seth P
Is it possible that the (primary key index (which is a composite index that begins with gvkey, and is the only index on the table) isn't being used because the the gvkey parameter is somehow passed as an integer rather than as a string? The first EXEC below is pretty much instantaneous, whereas

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Michael Bayer
On May 12, 2014, at 6:33 PM, Seth P spadow...@gmail.com wrote: Is it possible that the (primary key index (which is a composite index that begins with gvkey, and is the only index on the table) isn't being used because the the gvkey parameter is somehow passed as an integer rather than as

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Seth P
Yep, it's not a SQLAlchemy issue. The following code demonstrates the problem with direct pyodbc access. import pyodbc import time def print_timing(func): def wrapper(*arg): t1 = time.time() rows = func(*arg) t2 = time.time() print(%30s() len=%d, last=%s,

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Seth P
Looks like other people have encountered similar problems with indices being ignored by prepared sql statements: http://www.postgresql.org/message-id/43250afa.7010...@arbash-meinel.com. (If the diagnosis there is correct, then I'm guessing the server would use a unique index where all the

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Michael Bayer
On May 12, 2014, at 7:35 PM, Seth P spadow...@gmail.com wrote: Looks like other people have encountered similar problems with indices being ignored by prepared sql statements: http://www.postgresql.org/message-id/43250afa.7010...@arbash-meinel.com. (If the diagnosis there is correct, then

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Seth P
Fair enough. I'll take a look at pymssql, though I suspect it may be a SQL Server rather than a driver issue. On Monday, May 12, 2014 7:50:03 PM UTC-4, Michael Bayer wrote: On May 12, 2014, at 7:35 PM, Seth P spad...@gmail.com javascript: wrote: Looks like other people have encountered

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Seth P
pymssql produces the same results as pyodbc. So it looks like a SQL Server issue. On Monday, May 12, 2014 8:06:08 PM UTC-4, Seth P wrote: Fair enough. I'll take a look at pymssql, though I suspect it may be a SQL Server rather than a driver issue. On Monday, May 12, 2014 7:50:03 PM UTC-4,

Re: [sqlalchemy] engine.execute() slow with parameterized queries?

2014-05-12 Thread Michael Bayer
well or a FreeTDS issue, more likely, if that's what you're using. the SQL compiler has a parameter called literal_binds that will make it render a bound parameter as an inline string, but it only supports a few very basic types.As far as getting this parameter set for a general class of