Gurpreet Sachdeva wrote:
So That means blindly using threads on any process won't help!
It depends on what "help" means to you.
Help means to improve processing speed in achieving a particular task... *Help* here also means that I have a processor farm, how do I best use them to get maximum processing speed out of them...
And that's the crux of your problem.
Firstly, you've discovered that attempts to partition a task using threads won't work well with a compute-intensive algorithm, since multiple threads will simply contend against each other for CPU in a single process.
This is not assisted by Python's use of a global interpreter lock (GIL) to ensure thread-safety.
A thread can release the GIL, but typically it will do this only when it's involved in some blocking operation. This means that threading can be useful to speed up network operations, for example, but even then you might get a better speedup using explicitly asynchronous techniques based on non-blocking sockets.
In that case you definitely need to be looking at multiprocess algorithms if you are sticking with Python. Since each process has its own copy of the interpreter, they each also have their own GIL, and so the operating system will be able to schedule the processes in parallel on separate CPUs.
The example you listed isn't a good use of thread for performance
The example was a dummy one to become friendly with Threads and to understand the working/power of them... The example which will be finally used with threads take 5 Days to complete... I want to convert that in few hours!
Long Journey ahead...
Indeed, but an interesting one, no doubt. Good luck.
regards Steve -- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ Holden Web LLC +1 703 861 4237 +1 800 494 3119 -- http://mail.python.org/mailman/listinfo/python-list
