On 8 Feb, 10:39, Vishal <vsapr...@gmail.com> wrote: > Is it possible that the Python process, creates copies of the > interpreter for each thread that is launched, and some how the thread > is bound to its own interpreter ?
In .NET lingo this is called an 'AppDomain'. This is also how tcl works -- one interpreter per thread. I once had a mock-up of that using ctypes and Python'c C API. However, the problem with 'app domains' is that OS handles are global to the process. To make OS handles private, the easiest solution is to use multiple processes, which incidentally is what the 'multiprocessing' modules does (or just os.fork if you are on Unix). Most people would not consider 'app domains' to be a true GIL-free Python, but rather think of free threading comparable to .NET, Java and C++. However, removing the GIL will do no good as long as CPython uses reference counting. Any access to reference counts must be atomic (e.g. requiring a mutex or spinlock). Here we can imagine using fine- grained locking instead of a global interpreter lock. There is a second problem, which might not be as obvious: In parallel computing there is something called 'false sharing', which in this case will be incurred on the reference counts. That is, any updating will dirty the cache lines everywhere; all processors must stop whatever they are doing to synchronize cache with RAM. This 'false sharing' will put the scalability down the drain. To make a GIL free Python, we must start by removing reference counting in favour of a generational garbage collector. That also comes with a cost. The interpreter will sometimes pause to collect garbage. The memory use will be larger as well, as garbage remain uncollected for a while and is not immediately reclaimed. Many rely on CPython because the interpreter does not pause and a Python process has a small fingerprint. If we change this, we have 'yet another Java'. There are already IronPython and Jython for those who want this. Sturla -- http://mail.python.org/mailman/listinfo/python-list
