Re: [Python-Dev] Python parallel benchmark

2008-05-16 Thread Tom Pinckney
-0400, Tom Pinckney wrote: I found some other references where people were expressing concern over numpy releasing the GIL due to the fact that other C extensions could call numpy and unexpectedly have the GIL released on them (or something like that). Could you please post links to those? I

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Tom Pinckney
.8. On May 15, 2008, at 10:55 PM, Greg Ewing wrote: Tom Pinckney wrote: If I look at top while running 2 or more threads, both cores are being used 100% and there is no idle time on the system. If you run it with just one thread, does it use up only one core's worth of CPU? If so, this s

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Tom Pinckney
were expressing concern over numpy releasing the GIL due to the fact that other C extensions could call numpy and unexpectedly have the GIL released on them (or something like that). On May 15, 2008, at 6:43 PM, Nick Coghlan wrote: Tom Pinckney wrote: All the discussion recently about pyprocessin

[Python-Dev] Python parallel benchmark

2008-05-15 Thread Tom Pinckney
All the discussion recently about pyprocessing got me interested in actually benchmarking Python's multiprocessing performance to see if reality matched my expectations around what would scale up and what would not. I knew Python threads wouldn't be good for compute bound problems, but I wa

Re: [Python-Dev] Addition of "pyprocessing" module to standard lib.

2008-05-14 Thread Tom Pinckney
On May 14, 2008, at 12:32 PM, Andrew McNabb wrote: Think of the processing module as an alternative to the threading module, not as an alternative to MPI. In Python, multi-threading can be extremely slow. The processing module gives you a way to convert from using multiple threads to usi

Re: [Python-Dev] Addition of "pyprocessing" module to standard lib.

2008-05-13 Thread Tom Pinckney
Why not use MPI? It's cross platform, cross language and very widely supported already. And there're Python bindings already. On May 13, 2008, at 8:52 PM, Jesse Noller wrote: I am looking for any questions, concerns or benchmarks python-dev has regarding the possible inclusion of the pyproces

Re: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module

2008-05-12 Thread Tom Pinckney
Is there any thought to extending escape to escape / unescape to by default handle characters other than <, >, and &? At a minimum it should handle arbitrary &xxx; values. Ideally, it would also handle common other symbolic names besides < > etc. HTML from common web sites such as nytimes.c

Re: [Python-Dev] urllib unicode handling

2008-05-07 Thread Tom Pinckney
I was assuming urllib.quote/unquote would only be called on text intended to be used in non-hostname portions of the URIs. I'm not sure if this is the actual intent of urllib.quote and perhaps the documentation should be updated to specify what precisely it does and then peopel can decide w

Re: [Python-Dev] urllib unicode handling

2008-05-07 Thread Tom Pinckney
Maybe I didn't understand the RFC quite right, but it seemed like how to handle hostnames was left as a choice between IDNA encoding the hostname or replacing the non-ascii characters with dashes? I guess in practice IDNA is the right decision. Another part I wasn't clear on is whether urll

Re: [Python-Dev] urllib unicode handling

2008-05-07 Thread Tom Pinckney
I may be missing something, but it seems that RFC 3987 (which is about IRIs) basically says: 1) IRIs are identical to URIs except they may have unicode characters in them 2) IRIs must be converted to URIs before being used in HTTP 3) The way to convert IRIs to URIs is to UTF-8 encode the uni

[Python-Dev] urllib unicode handling

2008-05-06 Thread Tom Pinckney
Hi, While trying to use urllib in python 2.5.1 to HTTP GET content from various web sites, I've run into a problem with urllib.quote (and .quote_plus): they don't accept unicode strings. I see that this is an issue that has been discussed before: see this thread: http://mail.pytho