Re: [Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen

2014-03-28 Thread Richard Oudkerk
On 28/03/2014 06:35 pm, Josiah Carlson wrote: If it were me, I'd define three methods, with longer names to clarify what they do, e.g. proc.write_nonblocking(data) data = proc.read_nonblocking() data = proc.read_stderr_nonblocking() Easily doable. To implement

Re: [Python-Dev] .clinic.c vs .c.clinic

2014-01-18 Thread Richard Oudkerk
On 18/01/2014 05:09 pm, Antoine Pitrou wrote: Or, if this collides with Include/*, one of the following: memoryview_func.h // public functions memoryview_if.h // public interface Objects/memoryview.clinic.h should be fine. Or maybe have a __clinic__ directory similar to

Re: [Python-Dev] Accepting PEP 3154 for 3.4?

2013-11-18 Thread Richard Oudkerk
On 18/11/2013 10:25pm, Tim Peters wrote: The difference between 8 and 15 is, e.g., nearly doubling the amount of network traffic (for apps that use pickles across processes or machines). I tried using multiprocessing.Pipe() and send_bytes()/recv_bytes() to send messages between processes:

Re: [Python-Dev] Accepting PEP 3154 for 3.4?

2013-11-18 Thread Richard Oudkerk
On 19/11/2013 12:55am, Tim Peters wrote: [Richard Oudkerk] I tried using multiprocessing.Pipe() and send_bytes()/recv_bytes() to send messages between processes: 8 bytes messages -- 525,000 msgs/sec 15 bytes messages -- 556,000 msgs/sec So the size of small messages does not seem

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

2013-10-18 Thread Richard Oudkerk
On 18/10/2013 5:31pm, Guido van Rossum wrote: I'm working from home today and my Windows laptop is in the office, so I won't be able to test my latest Tulip changes on Windows (I just renamed pause to pause_reading, and hope to commit pause_reading later today). Is anyone luckier? $ hg id

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

2013-10-18 Thread Richard Oudkerk
On 18/10/2013 6:15pm, Guido van Rossum wrote: Thanks! There are some new changes (I fixed a race with sockets closing) and I hope to land flow control (finally) later today. Do you know what those skips are? I suspect they might be due to ssl not working for you either. :-( Lack of support

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

2013-10-18 Thread Richard Oudkerk
On 18/10/2013 6:57pm, Guido van Rossum wrote: Thanks! Those are all expected (though contributions are always welcome -- not looking at you specifically :-). Does examples/fetch3.py work for you with an https URL? (Try http://dropbox.com, i.e. without 's' -- you get two redirects to https URLs.

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

2013-10-18 Thread Richard Oudkerk
On 18/10/2013 9:19pm, Guido van Rossum wrote: Maybe the dummy socket returned by wrap_socket() is not acceptable for select? An error SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:553)') is being raised in _on_handshake(). This seems to result in the

Re: [Python-Dev] Looking for volunteers to test Tulip on Windows

2013-10-18 Thread Richard Oudkerk
On 18/10/2013 10:37pm, Guido van Rossum wrote: Good sleuthing! Does the attached patch fix it? (Off-topic: the code is pretty inconsistent about catching BaseException. Maybe it shouldn't be caught at all?) It fixes it in the sense of printing a sensible traceback;-) $

Re: [Python-Dev] cpython: Try doing a raw test of os.fork()/os.kill().

2013-10-17 Thread Richard Oudkerk
On 17/10/2013 5:35pm, Charles-François Natali wrote: For this kind of ad-hoc testing, you can also use a custom builder to avoid disrupting the main source tree: AFAICT, the problem he's trying to debug (issue #19227) only occurs on two specific - stable - buildbots. It looks like there are

Re: [Python-Dev] Add a transformdict to collections

2013-09-10 Thread Richard Oudkerk
On 10/09/2013 10:28am, Antoine Pitrou wrote: Therefore I propose adding the general pattern. Simple example: d = transformdict(str.lower) d['Foo'] = 5 d['foo'] 5 d['FOO'] 5 list(d) ['Foo'] I guess another example is creating an identity dict (see

Re: [Python-Dev] Add a transformdict to collections

2013-09-10 Thread Richard Oudkerk
On 10/09/2013 3:15pm, Armin Rigo wrote: Hi Richard, On Tue, Sep 10, 2013 at 3:42 PM, Richard Oudkerk shibt...@gmail.com wrote: I guess another example is creating an identity dict (see http://code.activestate.com/lists/python-ideas/7161/) by doing d = transformdict(id) This is bogus

Re: [Python-Dev] IOCP (I/O Completion Port) in Python ?

2013-09-10 Thread Richard Oudkerk
On 10/09/2013 11:58am, 张佩佩 wrote: Hello: I wondering why there is no standard IOCP module in Python ? As I know: Python3 have support epoll in linux and kqueue in freebsd. Is there a plan to add IOCP module for Windows ? _winapi does have some undocumented support for IOCP (used only

Re: [Python-Dev] windows file closing race condition?

2013-09-06 Thread Richard Oudkerk
On 06/09/2013 11:23am, Tim Golden wrote: On 06/09/2013 11:14, Antoine Pitrou wrote: Le Fri, 06 Sep 2013 08:58:06 +0100, Tim Golden m...@timgolden.me.uk a écrit : What should Python do? Maybe using FILE_SHARE_DELETE could help? http://bugs.python.org/issue15244 I don't think so. It's the

Re: [Python-Dev] windows file closing race condition?

2013-09-06 Thread Richard Oudkerk
On 06/09/2013 1:55pm, Paul Moore wrote: ... If you rename the file to anywhere but the same directory, you potentially have permission issues. Using the root directory avoids permission issues -- users always have write access there. -- Richard

Re: [Python-Dev] PEP 446: issue with sockets

2013-08-21 Thread Richard Oudkerk
On 21/08/2013 1:19am, Victor Stinner wrote: 2013/8/21 Guido van Rossum gu...@python.org: Also, are you sure the things returned by socket.fleno() are really Windows handles? I thought they were some other artificial namespace used just by sockets. (You know what? I know understand and love

Re: [Python-Dev] PEP 446: issue with sockets

2013-08-21 Thread Richard Oudkerk
On 21/08/2013 1:50pm, Victor Stinner wrote: 2013/8/21 Richard Oudkerk shibt...@gmail.com: On 21/08/2013 1:19am, Victor Stinner wrote: I don't know if a socket handle is similar to file handles or if they are specials. At least, GetHandleInformation() and SetHandleInformation() functions, used

Re: [Python-Dev] Green buildbot failure.

2013-08-11 Thread Richard Oudkerk
On 11/08/2013 11:00am, Antoine Pitrou wrote: You've got the answer at the bottom: program finished with exit code 0 So for some reason, the test suite crashed, but with a successful exit code. Buildbot thinks it ran fine. Was the test terminated because it took too long?

Re: [Python-Dev] Green buildbot failure.

2013-08-11 Thread Richard Oudkerk
http://stackoverflow.com/questions/2061735/42-passed-to-terminateprocess-sometimes-getexitcodeprocess-returns-0 -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable

2013-08-05 Thread Richard Oudkerk
On 06/08/2013 1:23am, Victor Stinner wrote: Each operating system handles the inheritance of file descriptors differently. Windows creates non-inheritable file descriptors by default, whereas UNIX creates inheritable file descriptors by default. The Windows API creates non-inheritable

Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable

2013-08-05 Thread Richard Oudkerk
On 06/08/2013 1:23am, Victor Stinner wrote: Each operating system handles the inheritance of file descriptors differently. Windows creates non-inheritable file descriptors by default, whereas UNIX creates inheritable file descriptors by default. The Windows API creates non-inheritable

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 7:44am, Charles-François Natali wrote: Then how about changing the default to creating file descriptors unheritable on Windows (which is apparently the default)? Then you can implement keep_fds by setting them inheritable right before creation, and resetting them right after: sure

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 12:59am, Victor Stinner wrote: On Windows, a file cannot be removed if at least one process opened it. If you create a temporary file, run a program, and delete the temporary file: the deletion fails if the program inherited the file and the program is not done before the deletion.

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 1:21am, Victor Stinner wrote: 2013/7/30 Victor Stinner victor.stin...@gmail.com: I would be nice to have a pass_handles on Windows. I'm not sure that it's possible to implement this atomically. It's probably better to leave the application to choose how the inheritance is

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 12:30pm, Victor Stinner wrote: Is it possible to implement atfork on Windows? On Windows the patch does expose atfork.getlock() and uses it in subprocess. (It should also modify os.spawn?(), os.startfile() etc.) But atfork.atfork() is Unix only. A Python lock would be

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Richard Oudkerk
On 30/07/2013 8:09am, Charles-François Natali wrote: If we don't want to inherit file descriptors in child processes, the answer is simple: the subprocess module (this fact is not even mentioned in the PEP). Note that on Windows subprocess has no equivalent of a passfds argument, and if you

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Richard Oudkerk
On 30/07/2013 11:52am, Victor Stinner wrote: You can redirect standard streams (stdin, stdout, stderr) using the startup info structure: startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES startupinfo.hStdInput = p2cread startupinfo.hStdOutput = c2pwrite startupinfo.hStdError = errwrite

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

2013-07-24 Thread Richard Oudkerk
Wow. Indeed you can -- I just tested this myself. How is this accomplished? I guess the CRT has a backchannel to talk to itself when it creates a process using spawn*? CreateProcess() takes a STARTUPINFO argument with undocumented fields cbReserved2, lpReserved2. They are used to pass an

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

2013-07-24 Thread Richard Oudkerk
On 24/07/2013 7:17pm, Guido van Rossum wrote: Does it also inherit sockets (which take up a different namespace than regular FDs in CRT, unlike UNIX)? Not reliably. Processes created with CreateProcess() seems to inherit socket handles just like normal handles on my computer, but on some

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

2013-07-24 Thread Richard Oudkerk
On 24/07/2013 10:50pm, Victor Stinner wrote: So would it be possible to implement the pass_fds parameter of subprocess using spawnl() or the undocumented fields? Not in a non-racy way. spawnv() calls CreateProcess() with bInheritHandles=TRUE, so *all* inheritable handles are inherited by the

Re: [Python-Dev] Inherance of file descriptor and handles on Windows (PEP 446)

2013-07-23 Thread Richard Oudkerk
On 23/07/2013 11:45pm, Victor Stinner wrote: Said differently: the HANDLE_FLAG_INHERIT flag only has an effect on *handles*, as indicated in its name. On Windows, file*descriptors* are never inherited (are always closed) in child processes. I don't think that it is possible to inherit file

Re: [Python-Dev] Dash

2013-07-18 Thread Richard Oudkerk
On 18/07/2013 7:55pm, Guido van Rossum wrote: This may well be a relic from times when in typewritten text there were really only two types of dashes: a short one, typed as a hyphen, used for both hyphen and en-dash (and minus sign), and a long one, typed as two hyphens, used for em-dash. I

Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores

2013-07-16 Thread Richard Oudkerk
On 16/07/2013 6:44am, Nick Coghlan wrote: Clarifying what constitutes an internal interface in a way that doesn't require renaming anything is a necessary prerequisite for bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes its internal implemetnation API as import pip rather

Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #18277: Document quirks of multiprocessing queue.

2013-06-24 Thread Richard Oudkerk
On 24/06/2013 7:30pm, A.M. Kuchling wrote: On Mon, Jun 24, 2013 at 03:53:27PM +0200, richard.oudkerk wrote: http://hg.python.org/cpython/rev/8dcc4e017d42 + but should not cause any pratical difficulties -- you can always ^^ practical + infinitessimal

Re: [Python-Dev] Problem with pydoc script in PEP 405 venvs

2013-06-17 Thread Richard Oudkerk
On 17/06/2013 5:27pm, Vinay Sajip wrote: 3. Rename the pydoc.py script to pydoc-script.py and introduce a simple .exe launcher pydoc.exe adjacent to it (which is how setuptools and distlib handle installed scripts). Can't a batch file pydoc.bat be used? -- Richard

[Python-Dev] Async subprocesses on Windows with tulip

2013-05-19 Thread Richard Oudkerk
Attached is a pretty trivial example of asynchronous interaction with a python subprocess using tulip on Windows. It does not use transports or protocols -- instead sock_recv() and sock_sendall() are used inside tasks. I am not sure what the plan is for dealing with subprocesses currently.

Re: [Python-Dev] Async subprocesses on Windows with tulip

2013-05-19 Thread Richard Oudkerk
On 19/05/2013 5:03pm, Benjamin Peterson wrote: Shouldn't this go to the python-tulip list? Yes. Sorry about that. -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] [RELEASED] Python 2.7.5

2013-05-19 Thread Richard Oudkerk
On 20/05/2013 12:47am, Pierre Rouleau wrote: Moreover, the development version is 3.4, and in Python 3 the int type is a variable-length integer type (sys.maxint doesn't exist anymore). So this discussion is largely moot now. Good to know. Too bad there still are libraries not

Re: [Python-Dev] PEP 442: Safe object finalization

2013-05-18 Thread Richard Oudkerk
On 18/05/2013 9:59am, Antoine Pitrou wrote: This PEP proposes to turn CI disposal into the following sequence (new steps are in bold): 1. Weakrefs to CI objects are cleared, and their callbacks called. At this point, the objects are still safe to use. 2. **The finalizers of all CI objects

Re: [Python-Dev] PEP 442: Safe object finalization

2013-05-18 Thread Richard Oudkerk
On 18/05/2013 3:18pm, Antoine Pitrou wrote: It works fine: $ ./python sbt.py __main__.Node object at 0x7f3acbf8f400 __main__.Node object at 0x7f3acbf8f878 __main__.Node object at 0x7f3acbf8f878 __main__.Node object at 0x7f3acbf8f400 The reason is that, when you execute del self.next, this

Re: [Python-Dev] PEP 4XX: pyzaa Improving Python ZIP Application Support

2013-05-06 Thread Richard Oudkerk
So the bug would just cause .pyzw files to be opened with py instead of pyw? Won't this be harmless? I think the worst that would happen would be that you get a redundant console window if you are not already running powershell inside a console. -- Richard

Re: [Python-Dev] relative import circular problem

2013-04-04 Thread Richard Oudkerk
On 04/04/2013 4:17pm, Guido van Rossum wrote: I don't really see what we could change to avoid breaking code in any particular case -- the burden is up to the library to do it right. I don't see a reason to forbid any of this either. How about having a form of relative import which only works

Re: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on Windows

2013-03-23 Thread Richard Oudkerk
On 23/03/2013 10:06am, Paul Moore wrote: One example of a non-system-wide installation is a source build of Python. PEP 405 venvs created from a source build should work in the same way as venvs created using an installed Python. Thanks. I hadn't thought of that case. However, I'm still not

Re: [Python-Dev] cffi in stdlib

2013-02-27 Thread Richard Oudkerk
On 27/02/2013 9:21am, Maciej Fijalkowski wrote: That's somehting that will have to be resolved before cffi can be included in the stdlib, fat binaries are supported by CPython and are used the binary installers. Ronald if cpython supports it and you can load it using dlopen, it does work

Re: [Python-Dev] Marking GC details as CPython-only

2013-02-13 Thread Richard Oudkerk
On 13/02/2013 7:25pm, Antoine Pitrou wrote: I think resurrecting objects from __del__ is crazy, so IMO what you suggest is fine. You mean like subprocess.Popen.__del__? I quite agree. -- Richard ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Release or not release the GIL

2013-02-01 Thread Richard Oudkerk
On 01/02/2013 1:44am, Guido van Rossum wrote: I'm guessing those system calls are just implemented by the kernel and cannot block for I/O, so it was deemed unnecessary to release the GIL around them. I don't mind changing that though, you can never know what happens when you make a system call.

Re: [Python-Dev] Inconsistent behaviour of methods waiting for child process

2013-01-22 Thread Richard Oudkerk
On 22/01/2013 12:16pm, Marcin Szewczyk wrote: The 1) case is very impractical. Is there any movement towards standardization of those 3? Am I missing something and there is a way to get more information from Process.join()? With Process.join(), it looks like version 2.6 raises OSError but

Re: [Python-Dev] test___all__ polluting sys.modules?

2012-12-31 Thread Richard Oudkerk
On 30/12/2012 9:52pm, Antoine Pitrou wrote: If you have an object with a __del__ method as a module global, the cyclic gc will refuse to consider the module globals at all (which means it will affect unrelated objects). So, yes, I think the shutdown purge is still necessary. Perhaps there are

Re: [Python-Dev] test___all__ polluting sys.modules?

2012-12-30 Thread Richard Oudkerk
On 30/12/2012 12:31am, Eli Bendersky wrote: Would it make sense to save the sys.modules state and restore it in test___all__ so that sys.modules isn't affected by this test? Deleting module objects can cause problems because the destructor replaces values in the globals dict by None. If

Re: [Python-Dev] Socket timeout and completion based sockets

2012-11-27 Thread Richard Oudkerk
On 27/11/2012 9:35am, Kristján Valur Jónsson wrote: This worries me: If the file handle is associated with a completion port, an I/O completion packet is not queued to the port if a synchronous operation is successfully canceled... I think you can only abort a synchronous operation if you

Re: [Python-Dev] Handling support for newer OS features at run time

2012-11-27 Thread Richard Oudkerk
On 27/11/2012 10:49pm, Trent Nelson wrote: Ideally, a Windows binary should make WSAPoll/select.poll() available if running on Vista or above, without impacting the ability to run on XP. I assume you can do something like int WSAAPI (*pWSAPoll)(WSAPOLLFD *, ULONG, INT);

Re: [Python-Dev] Socket timeout and completion based sockets

2012-11-26 Thread Richard Oudkerk
On 26/11/2012 11:49am, Kristján Valur Jónsson wrote: However, other implementations of python sockets, e.g. ones that rely on IO completion, may not have the luxury of using select. For example, on Windows, there is no way to abort an IOCP socket call, so a timeout must be implemented by

Re: [Python-Dev] performance of {} versus dict(), de fmd(**kw): return kw trumps all ; -)

2012-11-15 Thread Richard Oudkerk
On 15/11/2012 4:21pm, Terry Reedy wrote: I was thinking that CPython could check the ref count of the input keyword dict to determine whether it is newly created and can be returned or is pre-existing and must be copied. But it seems not so. def d(**x): return sys.getrefcount(x) import sys

Re: [Python-Dev] performance of {} versus dict()

2012-11-14 Thread Richard Oudkerk
On 14/11/2012 4:23pm, Serhiy Storchaka wrote: PEP 8 recommends: a_dict = dict( x=1, y=2, z=3, ... ) and a_dict = { 'x': 1, 'y': 2, 'z': 3, ... } In which section? I can't see such a recommendation. -- Richard

Re: [Python-Dev] performance of {} versus dict()

2012-11-14 Thread Richard Oudkerk
On 14/11/2012 5:02pm, Xavier Morel wrote: In which section? I can't see such a recommendation. Whitespace in Expressions and Statements Other Recommendations 3rd bullet: — Don't use spaces around the = sign when used to indicate a keyword argument or a default parameter value. Oops, I

Re: [Python-Dev] Memoryviews should expose the underlying memory address

2012-09-20 Thread Richard Oudkerk
On 20/09/2012 5:53pm, David Beazley wrote: How? I must be missing something very obvious. I would not call it obvious, but you can do m = memoryview(bytearray(5)) ctypes.addressof(ctypes.c_char.from_buffer(m)) 149979304 However, this only works for writable memoryviews. For

Re: [Python-Dev] Unbinding of methods

2012-07-19 Thread Richard Oudkerk
On 19/07/2012 7:54pm, Antoine Pitrou wrote: Instead of a specific opcode, can't you use a suitable __reduce__ magic (or __getnewargs__, perhaps)? We want to limit the number of opcodes except for performance-critical types (and I don't think bound methods are performance-critical for the

[Python-Dev] Cloning threading.py using processes

2006-11-27 Thread Richard Oudkerk
Version 0.10 of the 'processing' package is available at the cheeseshop: http://cheeseshop.python.org/processing It is intended to make writing programs using processes almost the same as writing programs using threads. (By importing from 'processing.dummy' instead of 'processing' one can

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Richard Oudkerk
On 10/10/06, Fredrik Lundh [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Presumably with this library you have created, you have also written a fast object encoder/decoder (like marshal or pickle). If it isn't any faster than cPickle or marshal, then users may bypass the module and opt

Re: [Python-Dev] Cloning threading.py using proccesses

2006-10-12 Thread Richard Oudkerk
On 10/10/06, Josiah Carlson [EMAIL PROTECTED] wrote: the really interesting thing here is a ready-made threading-style API, I think. reimplementing queues, locks, and semaphores can be a reasonable amount of work; might as well use an existing implementation. Really, it is a matter of