"Chris Mellon" <[EMAIL PROTECTED]> writes: > > The compute intensive stuff (image rendering and crunching) has > > already had most of those skerricks pulled out. It is written in C > > and assembler > That means that this part is also unaffected by the GIL.
Right, it was a counterexample against the "speed doesn't matter" meme, not specifically against the GIL. And that code is fast because someone undertook comparatively enormous effort to code it in messy, unsafe languages instead of Python, because Python is so slow. > At the really high levels of scalability, such as across a server > farm, threading is useless. The entire point of threads, rather than > processes, is that you've got shared, mutable state. A shared nothing > process (or Actor, if you will) model is the only one that makes sense > if you really want to scale because it's the only one that allows you > to distribute over machines. The fact that it also scales very well > over multiple cores (better than threads, in many cases) is just > gravy. In reality you want to organize the problem so that memory intensive stuff is kept local, and that's where you want threads, to avoid the communications costs of serializing stuff between processes, either between boxes or between cores. If communications costs could be ignored there would be no need for gigabytes of ram in computers. We'd just use disks for everything. As it is, we use tons of ram, most of which is usually twiddling its thumbs doing nothing (as DJ Bernstein put it) because the cpu isn't addressing it at that instant. The memory just sits there waiting for the cpu to access it. We actually can get better-than-linear speedups by designing the hardware to avoid this. See: http://cr.yp.to/snuffle/bruteforce-20050425.pdf for an example. > The only hard example I've seen given of the GIL actually limiting > scalability is on single server, high volume Django sites, and I don't > think that the architecture of those sites is very scalable anyway. The stuff I'm doing now happens to work ok with multiple processes but would have been easier to write with threads. -- http://mail.python.org/mailman/listinfo/python-list