that's a lot of effort there. How confident are you that memory and references are handled correctly in the .c code? That's a lot of C code, and it took years for us to iron out all the memory leaks in the existing C extensions that we had - the original author eventually stopped maintaining them, and I had to take it all on myself and spend weeks learning the code and ironing out remaining, subtle issues (like http://hg.sqlalchemy.org/sqlalchemy/rev/8326 and http://hg.sqlalchemy.org/sqlalchemy/rev/8140). These are very insidious issues as they can't be diagnosed by usual gc reference counting.
On May 6, 2013, at 12:11 PM, Claudio Freire <klaussfre...@gmail.com> wrote: > On Mon, May 6, 2013 at 1:10 PM, Claudio Freire <klaussfre...@gmail.com> wrote: >> On Thu, May 2, 2013 at 3:34 PM, Claudio Freire <klaussfre...@gmail.com> >> wrote: >>> Without the C extension: >>> ncalls tottime percall cumtime percall filename:lineno(function) >>> 20811734 27.829 0.000 27.855 0.000 attributes.py:171(__get__) >>> 7631984 13.532 0.000 31.851 0.000 ruby.py:86(get_param) >>> >>> With the C extension: >>> ncalls tottime percall cumtime percall filename:lineno(function) >>> 7631984 19.514 0.000 21.051 0.000 ruby.py:86(get_param) >>> >>> Notice how the C extension saves a total of 10s (cumtime, sum of internal >>> and external time). >>> >>> There's no DB access when hitting those arguments, as everything has been >>> eagerly loaded. It's all function call overhead. >>> >>> Assuming an application makes heavy use of attributes, as get_param does >>> (expectable of straightforward code I'd think), that's a 30% speedup of >>> CPU-bound code. >>> >>> As soon as I get GC right I'll post the patch. >> >> >> So... I got GC right (I think). I had to remove a few lines from >> profiles.txt because, obviously, there's a lot less function calls >> now. >> >> There's a second patch, that adds __slots__ to instance state. I found >> it speeds up things, marginally, but consistently (State.__init__ was >> another function weighing a lot because of millions of calls, this is >> the only way I found to speed it up). >> >> I'll get around to the Py3 things now. >> >> PS: Sorry I based it on 0.7.10... my app runs on that... I imagine I >> could upgrade to 0.8 with little effort, but never got around to >> actually doing it. > > > Stupid me... forgot to attach them. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > <SQLAlchemy-0.7.10-cinstrumented.patch><SQLAlchemy-0.7.10-slotstate.patch> -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
