On 8 Sep, 04:22, ganesh <ganeshbo...@gmail.com> wrote: > My application is a TCP server having multiple client connectons. C++ > PTHREADS are for each connected socket and the message received on the > socket is evaluated by python functions. > If I use only one process level python interpreter, then every thread > has to lock the GIL & so blocking the other threads from executing the > python code even if it is not the same python function that locking > thread is calling.
Usually, TCP-servers are I/O bound. You can safely use a single Python process for this. A function evaluating a request will hold the GIL for a while (but not until it's done). But most other threads will be blocked waiting for I/O. Thus, there will be little contention for the GIL anyway, and it should not affect scalability much. Only when multiple requests are processed simultaneously will threre be contention for the GIL. You can create high-performance TCP servers in plain Python using e.g. Twisted. If you are in the strange situation that a TCP server is compute-bound, consider using multiple processes (os.fork or multiprocessing). > I think there is no way that we can achieve this because of the GIL > being a process level state. Least I can do is have one python > interpreter initialized in main thread and lock the GIL in every > thread for python calls. I think you will find out your server is indeed I/O bound, like 99.9% or all other TCP servers on this planet. Try to embed a single interpreter first. Use the simplified GIL API I showed you. Most likely you will find that it suffice. If you need something more scalable, associate each pthread with a separate Python process - e.g. using a named pipe on Windows or Unix domain socket on Linux. -- http://mail.python.org/mailman/listinfo/python-list