This is the ORM affecting 124K statements so must be a very large data persist (it seems like a persist heavy operation, i see 287K objects total treated as part of units of work).
It seems like you are calling commit() a very large number of times. So the most obvious enhancement would be to call this a whole lot less - the commit necessitates a flush, and also by default fully expires the session (unless you turn off expire_on_commit), meaning all rows have to be fully reloaded, which is probably making the number of statements executed much larger than it needs to be. You might want to establish better control over when flush() is called. Sometimes turning off autoflush and having it flush the whole thing just once, or at some pre-determined interval within a transaction, spans out the work more efficiently. A good deal of time is spent in compiling constructs into SQL strings here, there is a feature whereby this can be cut down dramatically for similar statements executed many times which is the "compiled_cache" execution option. The ORM uses this a bit internally now though it might be a nice feature for you to be able to switch it on for a given session, and have all SQL statement compilation cached for the life of that session. This feature can be approximated right now though I'd want to modify _save_obj to not overwrite the cache with its own, which defeats the usage of a session-spanning compilation cache. the last thing there is the only further potential optimization to SQLA that is apparent, and would probably cut at least a few million function calls out of the run you have here. Other than that you're already working with the most optimized version of the ORM :). On May 28, 2010, at 4:14 PM, Jason Baker wrote: > I'm attaching output from cProfile from our application. I'm mainly sending > this to you for two reasons: > > 1. In case profiling data is helpful to you guys for optimizing SQLAlchemy. > 2. Because I'm looking for ways to speed my application up, and I am not > sure how to interpret the SQLAlchemy portions. > > It looks like the biggest timesink is Session.commit. Does anyone have any > general advice on making commit go faster? I can answer any questions that > aren't answered by the profiling data, but I'm not sure how much actual > source code I can give out. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > <a0db256c-98fa-4ae6-a8cd-26568c60fb9frlproc.prof> -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
