Jonathan Ellis wrote: > I'm working on an application that makes heavy use of Queue objects in > a multithreaded environment. > > By "heavy" I mean "millions of calls to put and get, constituting ~20% > of the app's run time." The profiler thinks that a significant amount > of time is spent in this code -- not just a consumer waiting for a > producer, but actual _empty, notify calls, etc.
I wonder if the use case would support hand-crafting an alternative. Queues appear fairly heavy weight (when you look at the implementation), and while they are very robust, if your own needs allow putting many items in at the same time, or getting many items out, for example, then perhaps you could come up with a much faster alternative based on the primitives (e.g. Event, Condition, etc) and perhaps some wrapped data structure other than the list that Queues use. -Peter -- http://mail.python.org/mailman/listinfo/python-list