Re: [sqlalchemy] timing / profiling object loading ?

2013-09-27 Thread Jonathan Vanasco
funny- I never noticed the time column in pyramid_debugtoolbar -- I use it all the time ! i ended up doing a lot of my profiling outside of pyramid though. that's something i really love about sqlalchemy (and pyramid) - once the models were built, I could just import them into test scripts and

RE: [sqlalchemy] timing / profiling object loading ?

2013-09-27 Thread Jeff Peck
On Thursday, September 26, 2013 11:58:26 AM UTC-4, Michael Bayer wrote: That's why I don't have a caching function included with SQLAlchemy. Because then I'd be debugging it, not you :) Ha! My caching is pretty lightweight. I do need to figure out a better system though -- that's for post

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-26 Thread Jonathan Vanasco
On Thursday, September 26, 2013 11:58:26 AM UTC-4, Michael Bayer wrote: > > That's why I don't have a caching function included with SQLAlchemy. > Because then I'd be debugging it, not you :) > Ha! My caching is pretty lightweight. I do need to figure out a better system though -- that's fo

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-26 Thread Michael Bayer
That's why I don't have a caching function included with SQLAlchemy. Because then I'd be debugging it, not you :) On Sep 26, 2013, at 11:38 AM, Jonathan Vanasco wrote: > Thanks. > > I caved in and stayed up until 3am last night to add a ton of log.debug() > statements across the app. > >

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-26 Thread Jonathan Vanasco
Thanks. I caved in and stayed up until 3am last night to add a ton of log.debug() statements across the app. The culprit was my read-through-caching layer ( built on dogpile ). It was implemented in such a way that SqlAlchemy looked to have issues. The performance on the DBM datatstore was

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-26 Thread Michael Bayer
the profiling will show you if there's some network/server overhead from fetching rows, if you can see where psycopg2 is actually doing that. On Sep 26, 2013, at 2:44 AM, Jonathan Vanasco wrote: > yeah. painfully slow. > > i can't seem to recreate this on a test script. it happens every so o

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-25 Thread Jonathan Vanasco
yeah. painfully slow. i can't seem to recreate this on a test script. it happens every so often in a pyramid app , but I can't recreate it on a bootstrapped (command line) pyramid instance. my test suite shows this happening instantly. this has been troubling me for over a week since i notic

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-25 Thread Michael Bayer
that seems *very* slow? On Sep 25, 2013, at 5:05 PM, Jonathan Vanasco wrote: > very good. thanks. > > i accidentally left a LIMIT off a query. The DB was optimized to return the > results in about .003s ; but it seems to have taken about 2 minutes for > sqlalchemy to generate ~1300 objects

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-25 Thread Jonathan Vanasco
very good. thanks. i accidentally left a LIMIT off a query. The DB was optimized to return the results in about .003s ; but it seems to have taken about 2 minutes for sqlalchemy to generate ~1300 objects from the rows. -- You received this message because you are subscribed to the Google Grou

Re: [sqlalchemy] timing / profiling object loading ?

2013-09-25 Thread Michael Bayer
I tend to use standard python profiling for that http://stackoverflow.com/questions/1171166/how-can-i-profile-a-sqlalchemy-powered-application/1175677#1175677 On Sep 25, 2013, at 4:44 PM, Jonathan Vanasco wrote: > is there a way to time the time spent 'loading' an object from the database ? >

[sqlalchemy] timing / profiling object loading ?

2013-09-25 Thread Jonathan Vanasco
is there a way to time the time spent 'loading' an object from the database ? i think there might be a bottleneck in a part of my application that is related to the object instantiation. just to be clear , in this flow: a) `query.all()` b) sqlalchemy compiles query c) sqlalchemy to db