On Mon, Nov 04, 2013 at 06:02:47PM -0800, Danny Yoo wrote: > To visualize the sheer scale of the problem, see: > > http://i.imgur.com/X1Hi1.gif > > which would normally be funny, except that it's not quite a joke. :P
Nice visualisation! Was that yours? > So you want to minimize hard disk usage as much as possible. "Thrashing" > is precisely the situation you do not want to have when running a large > analysis. Yes, thrashing is a disaster for performance. But I think that hard drive latency fails to demonstrate just how big a disaster. True, hard drive access time is about 100 times slower than RAM. But that's just a constant scale factor. What really, really kills performance with thrashing is that the memory manager is trying to move chunks of memory around, and the overall "move smaller blocks of memory around to make space for a really big block" algorithm ends up with quadratic (or worse!) performance. -- Steven _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
