On 2008-05-21, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > I'd appreciate any help. I've got a list of files in a directory, and > I'd like to iterate through that list and process each one. Rather > than do that serially, I was thinking I should start five threads and > process five files at a time. > > Is this a good idea? I picked the number five at random... I was
Depends what you are doing. If you are mainly reading/writing files, there is not much to gain, since 1 process will already push the disk IO system to its limit. If you do a lot of processing, then more threads than the number of processors is not much use. If you have more 'burtsy' behavior (first do lot of reading, then lot of processing, then again reading, etc), then the system may be able to do some scheduling and keep both the processors and the file system busy. I cannot really give you advice on threading, I have never done that. You may want to consider an alternative, namely multi-tasking at OS level. If you can easily split the files over a number of OS processes (written in Python), you can make the Python program really simple, and let the OS handle the task-switching between the programs. Sincerely, Albert -- http://mail.python.org/mailman/listinfo/python-list
