Well I guess I outsmarted myself on this one. After implementing my object hasher using dict, my code slowed down by nearly a factor of 10 LOL.
The hashing code itself shows up as a minimal contribution to the overall time in the profiler. However it does require a little extra logic in the initializer and a copy of a few pointers. Basically I copy all the C device pointers to a new cython class which then get's saved in a dictionary upon the deallocation of a matrix object. If that object exists in the dictionary it is given to the allocator instead of allocating a new block. I'm wondering just what kind of overhead Im up against with regards to the python initialization of my cython defined class? This almost suggests that the overhead problem is actually in python/cython and not just in my C code. -Matt Robert Bradshaw <rober...@...> writes: > > On Jan 7, 2010, at 12:05 AM, Stefan Behnel wrote: > > > Matthew Bromberg, 06.01.2010 21:50: > >> How does tuple or list compare speed wise with dict? > > > > Like apples and oranges, basically. > > If you're trying to index into it with an int, especially a c int, > lists and tuples will be much faster. > > >> Ultimately I have to hash into my list using size information. > > > > Any specific reason why you /can't/ use a dict? > > Which will likely be just as fast, if not faster, than hashing into a > list manually yourself. > > >> This also still does not address my confusion with regards to how to > >> capture a python object before it get's destroyed. > > > > As long as there is a reference to it (e.g. in the hash table), it > > won't > > get deallocated. So: use a Python list for your hash table, stop > > caring > > about ref-counts and it will just work. > > +1. Ideally, you should never have to worry about reference counts > when working with Cython at all. > > I'm still not quite sure exactly what you're trying to do, but if it's > creating and deleting thousands of these objects a second and that's > killing you (the actual allocation/deallocation, not the > initialization) then what you might want to do is something like > > http://hg.sagemath.org/sage-main/file/21efb0b3fc47/sage/rings/real_double.pyx#l2260 > > which is a bit hackish and will probably need to be adapted to your > specific case. If initializing is expensive, than you can probably > keep around a pool of initalized pointers/buffers/whatever, and have > the object creation just set/unset these fields (much cleaner). > > - Robert > _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
