Gotcha. Slightly weird that threading.local would fix that. It sounds like what you may have potentially just done is create N MySQL clients, one-per each thread (resulting in each thread just blocking on its own query, rather than each thread blocking on getting access to that same MySQL connection when someone else is using it).
Out of curiosity, does "causing error responses for other clients" mean causing timeouts? Or was there actually some kind of error being generated from the internals? -----Original Message----- From: Phillip B Oldham [mailto:[email protected]] Sent: Friday, August 06, 2010 6:08 AM To: [email protected] Subject: Re: Short question re. threading with a Python server Seems my error was due to threading, specifically with the MySQL client. I had to alter the handler class so that it extended `threading.local` otherwise a MySQL query would block the whole process, causing error responses for other clients. On Tue, Aug 3, 2010 at 2:41 PM, Phillip B Oldham <[email protected]> wrote: > Thanks Mark, that's really helpful. We're seeing a very odd bottleneck > and thought it could've been down to issues with threading in our > Python server. We've no shared state except what is persisted through > memcached and MySQL, so it mustn't be that. I'll continue to dig > around. > > Cheers! > > On Mon, Aug 2, 2010 at 8:57 PM, Mark Slee <[email protected]> wrote: >> Generally, no. Python's threading.local is a specific way of offering >> thread-local storage. >> >> You should NOT use threading.local if you're trying to safely share data >> between threads. That's not what it does -- rather it actually exposes a >> *different* copy of the data to each thread. >> >> So, you should use it if you actually want N copies of some data (where N is >> the number of threads), but you want a simple interface to refer to them all >> by the same name (with the clear understanding that you are actually >> accessing N different pieces of data). >> >> There are certainly situations in which this can be useful in a threaded >> Thrift server, but it's by no means a general rule. >> >> -----Original Message----- >> From: Phillip B Oldham [mailto:[email protected]] >> Sent: Friday, July 30, 2010 2:39 AM >> To: thrift-user >> Subject: Short question re. threading with a Python server >> >> Should Python classes that are run via the >> TThreaded/TThreadPool/TNonBlocking server inherit from >> `threading.local`, in order to be thread safe? >> >> -- >> Phillip B Oldham >> [email protected] >> +44 (0) 7525 01 09 01 >> > > > > -- > Phillip B Oldham > [email protected] > +44 (0) 7525 01 09 01 > -- Phillip B Oldham [email protected] +44 (0) 7525 01 09 01
