New submission from Nick Edds <[EMAIL PROTECTED]>: Here is a working, multiprocess version of 2to3 with a few caveats. First, you need to already have the processing module installed for this to work. If we don't want to include processing in some way, I think I can modify this to only import processing and use the Process method if the user wants to run it with more than one process. Also, I know that logger is supposed to be thread safe, so I am correct in assuming that this means it is also multi-process safe? This fix delegates the fixing of files to a designated number of processes, so on a multi-core machine, it is significantly faster. It may be appropriate to add a test suite for it, but I have not yet done so. I've tested it on my dual-core laptop running Ubuntu and a quad-core mac and it appears to be working fine. I don't know if the use of tempfile was the right choice, but it seemed reasonable. Another possibility is to instead using a processing Queue and pass it the output, filename, and input rather than calling write_file. Also, the join timeout I used is completely arbitrary because I don't really have a sense of what a reasonable value for it should be.
---------- assignee: collinwinter components: 2to3 (2.x to 3.0 conversion tool) files: multiProcess.diff keywords: patch messages: 70277 nosy: collinwinter, nedds severity: normal status: open title: Multi-process 2to3 type: performance Added file: http://bugs.python.org/file10985/multiProcess.diff _______________________________________ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3448> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
