Simon Strobl <[EMAIL PROTECTED]> wrote: > Well, as I was using Python, I did not expect to have to care about > the language's internal affairs that much. I thought I could simply do > always the same no matter how large my files get. In other words, I > thought Python was really scalable.
It's not Python here. It's just how computers work. IMHO having a gargantuan dictionary in memory is not a good idea (unless explicitly proven otherwise): this is the kind of job databases have been created for. Besides, this is not a matter of Python. If you were using C or another language, I would have sugested to use databases in order to manipulate GB's of data. Luckily enought, using databases in Python is far easier than doing so in C/C++/Java. *And* there are thin abstractions over databases so you don't even need to know how to use them (though, I suggest that you *do* learn something about DB's and expecially relational DB's, SQL is not *that* bad). -- -riko -- http://mail.python.org/mailman/listinfo/python-list