Re: [Python-3000] Odd output from test -- buffering bug?
Thinking about this some more, the io module isn't thread-safe. It
probably should be (the old file objects were more-or-less
thread-safe, although I believe there might've been corner cases if
one thread were to close a file).
--Guido
2007/10/27, Bill Janssen <[EMAIL PROTECTED]>:
> No, not unless the test harness uses it. But there are two threads.
>
> > Hard to say. Never seen this before. Are you using fork() *anywhere*
> > in your tests (not necessarily the affected test)?
> >
> > 2007/10/27, Bill Janssen <[EMAIL PROTECTED]>:
> > > I'm seeing a sort of odd thing going on when running one of my tests.
> > > I'm seeing two lines of output, from two different threads, being
> > > duplicated when I run with "regrtest -u all -v test_ssl". This is
> > > with the latest 3K sources on PPC OS X 10.4.10.
> > >
> > > testSTARTTLS (test.test_ssl.ThreadedTests) ...
> > > client: sending b'msg 1'...
> > > [EMAIL PROTECTED]: sending b'msg 1'...
> > > server: new connection from ('127.0.0.1', 52371)
> > > server: new connection from ('127.0.0.1', 52371)
> > >
> > > This is output to an Emacs shell buffer, so it shows control
> > > characters in the output, and I'm seeing a NUL character being output
> > > there at the beginning of the third line. Both of the duplicated lines
> > > are being output with code like this:
> > >
> > > if test_support.verbose:
> > > sys.stdout.write(
> > > " client: sending %s...\n" % repr(msg))
> > >
> > > This looks like some kind of buffering bug. Is it in the test
> > > harness, or the standard I/O library?
> > >
> > > Bill
> > >
> > > ___
> > > Python-3000 mailing list
> > > Python-3000@python.org
> > > http://mail.python.org/mailman/listinfo/python-3000
> > > Unsubscribe:
> > > http://mail.python.org/mailman/options/python-3000/guido%40python.org
> > >
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-3000 mailing list
Python-3000@python.org
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] bug in i/o module buffering?
__index__() converts an "int-like" object to an int. This is needed to make sure that e.g. numpy integral scalars can be used for indexing. For a regular int it doesn't matter, so here it's a red herring. I'm asking about b because the error message "TypeError: 'slice' object does not support item deletion" would suggest that b is a slice object. I agree that doesn't sound very likely given the code though... :-( Could you step through this using pdb and investigate some more? Perhaps there's a refcount error somewhere in the C code? --Guido 2007/10/27, Bill Janssen <[EMAIL PROTECTED]>: > From RawIOBase.read(). What's __index__() do? > > b = bytes(n.__index__()) > > > More interesting is, what's b? > > > > 2007/10/27, Bill Janssen <[EMAIL PROTECTED]>: > > > In the following, 'n' is equal to 0 (read from a non-blocking socket). > > > Is this a bug in the I/O module buffering? > > > > > > Bill > > > > > > Traceback (most recent call last): > > > File "/local/python/3k/src/Lib/SocketServer.py", line 222, in > > > handle_request > > > self.process_request(request, client_address) > > > File "/local/python/3k/src/Lib/SocketServer.py", line 241, in > > > process_request > > > self.finish_request(request, client_address) > > > File "/local/python/3k/src/Lib/SocketServer.py", line 254, in > > > finish_request > > > self.RequestHandlerClass(request, client_address, self) > > > File "/local/python/3k/src/Lib/SocketServer.py", line 522, in __init__ > > > self.handle() > > > File "/local/python/3k/src/Lib/BaseHTTPServer.py", line 330, in handle > > > self.handle_one_request() > > > File "/local/python/3k/src/Lib/BaseHTTPServer.py", line 313, in > > > handle_one_request > > > self.raw_requestline = self.rfile.readline() > > > File "/local/python/3k/src/Lib/io.py", line 391, in readline > > > b = self.read(nreadahead()) > > > File "/local/python/3k/src/Lib/io.py", line 377, in nreadahead > > > readahead = self.peek(1, unsafe=True) > > > File "/local/python/3k/src/Lib/io.py", line 778, in peek > > > current = self.raw.read(to_read) > > > File "/local/python/3k/src/Lib/io.py", line 455, in read > > > del b[n:] > > > TypeError: 'slice' object does not support item deletion > > > > > > > > > ___ > > > Python-3000 mailing list > > > Python-3000@python.org > > > http://mail.python.org/mailman/listinfo/python-3000 > > > Unsubscribe: > > > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
2007/10/28, Brett Cannon <[EMAIL PROTECTED]>: > On 10/28/07, James Thiele <[EMAIL PROTECTED]> wrote: > > PEP 361 lists __bool__ support as being possible for 2.6 backporting. > > As of today the trunk build uses __nonzero__ like 2.5 but 3.0 alpha > > uses __bool__. Has a decision been made on whether this will make the > > cut for 2.6? > > > > In a more general vein, is there a cutoff date for producing a list of > > 3.0 features which will be backported to 2.6? > > Backporting decisions have not been made as the feature set of 3.0 is > still a moving target. Once we nail down the features (I am going to > guess not until b1 at the earliest) then backporting will probably > start. In this case, like many, the backport can't be an exact copy of the 3.0 code: 2.6 *must* support __nonzero__. But it should also support __bool__ as a fallback. I think it would be great if someone submited a patch to implement this (though it isn't necessarily the highest backporting priority). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] plat-mac seriously broken?
2007/10/27, Bill Janssen <[EMAIL PROTECTED]>: > > ISTR much of the plat-mac stuff was generated by Tools/bgen. If so, that > > would be the place to fix things. > > Sure looks like generated code. Be nice if that generator was run > during the build process, on OS X. That way you'd be sure to get code > that matches the platform and codebase. ISTR that the generator needs a lot of hand-holding. Fixing it would be A Project. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
2007/10/29, Fred Drake <[EMAIL PROTECTED]>: > On Oct 29, 2007, at 2:40 PM, James Thiele wrote: > > So just to clarify: > > 2.6 __nonzero__ first, then __bool__ (if patch submitted) > > 3.x __bool__ first, then __nonzero__ > > I'd expect switching the order for this to be a bug magnet. I'd much > rather see: > > 2.5 __nonzero__ only > 2.6 __bool__ first, then __nonzero__ (if patch submitted) > 3.x __bool__ first, then __nonzero__ > > The fewer variations there are in the algorithm, the better. Makes sense, if you change the 3.x rule to 3.x __bool__ only. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
2007/10/28, Bill Janssen <[EMAIL PROTECTED]>: > > Bill Janssen wrote: > > > that whole mess of code is a good argument for *not* exposing the > > > fileno in Python > > > > Seems to me that a socket should already *be* a file, > > so it shouldn't need a makefile() method and you > > shouldn't have to mess around with filenos. That model fits TCP/IP streams just fine, but doesn't work so well for UDP and other odd socket types. The assumption that "s.write(a); s.write(b) is equivalent to s.write(a+b)", which is fundamental for any "stream" abstraction, just doesn't work for UDP. Ditto for reading: AFAIK recv() truncates the rest of an UDP packet. > I like that model, too. I also wish the classes in io.py were sort of > inverted; that is, I'd like to have an IOStream base class with read() > and write() methods (and maybe close()), which things like Socket > could inherit from. FileIO would inherit from IOStream and from > Seekable, and add a fileno() method and "name" property. And so > forth. But apparently that's out; maybe in Python 4000. Actually, I'm still up for tweaks to the I/O model if it solves a real problem, as long as most of the high-level APIs stay the same (there simply is too much code that expects those to behave a certain way). I don't quite understand what you mean by inverted though. > Right now the socket is very much like an OS socket; with "send" and > "recv" being the star players, not "read" and "write". socket.makefile > wraps a buffered file-like interface around it. I was going to say "we can just replace SocketIO with a non-seekable _fileio.FileIO instance" until I realized that on Windows, socket fds and filesystem fds live in different spaces and are managed using different calls. That may also explain why the inversion you're looking for doesn't quite work (IIUC what you meant). The real issue seems to be file descriptor GC. Maybe we haven't written down the rules clearly enough for when the fd is supposed to be GC'ed, when there are both a socket and a SocketIO (or more) referencing it; and whether a close() call means something beyond dropping the last reference to the object. Or maybe we haven't implemented the rules right? ISTM that the SocketCloser class is *intended* to solve these issues. Back to your initial mail (which is more relevant than Greg Ewing's snipe!): > I think that the SocketCloser (new in Py3K) was developed to address > another issue, which is that there's a lot of library code which > assumes that the Python socket instance is just window dressing over > an underlying system file descriptor, and isn't important. In fact, > that whole mess of code is a good argument for *not* exposing the > fileno in Python (perhaps only for special cases, like "select"). > Take httplib and urllib, for instance. HTTPConnection creates a > "file" from the socket, by calling socket.makefile(), then in some > cases *closes* the socket (thereby reasonably rendering the socket > *dead*), *then* returns the "file" to the caller as part of the > response. urllib then takes the response, pulls the "file" out of it, > and discards the rest, returning the "file" as part of an instance of > addinfourl. Somewhere along the way some code should call "close()" > on that HTTPConnection socket, but not till the caller is finished > using the bytes of the response (and those bytes are kept queued up in > the real OS socket). Ideally, GC of the response instance should call > close() on the socket instance, which means that the instance should > be passed along as part of the response, IMO. Hm, I think you're right. The SocketCloser class wasn't written with the SSL use case in mind. :-( I wonder if one key to solving the problem isn't to make the socket *wrap* a low-level _socket instance instead of *being* one (i.e. containment instead of subclassing). Then the SSL code could be passed the low-level _socket instance and the high(er)-level socket class could wrap either a _socket or an SSL instance. The SocketCloser would then be responsible for closing whatever the socket instance wraps, i.e. either the _socket or the SSL instance. Then we could have any number of SocketIO instances *plus* at most one socket instance, and the wrapped thing would be closed when the last of the higher-level things was either GC'ed or explicitly closed. If you wanted to reuse the _socket after closing the SSL instance, you'd have to wrap it in a fresh socket instance. Does that make sense? (Please do note the difference throughout between _socket and socket, the former being defined in socketmodule.c and the latter in socket.py.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
On Oct 29, 2007, at 2:40 PM, James Thiele wrote: > So just to clarify: > 2.6 __nonzero__ first, then __bool__ (if patch submitted) > 3.x __bool__ first, then __nonzero__ I'd expect switching the order for this to be a bug magnet. I'd much rather see: 2.5 __nonzero__ only 2.6 __bool__ first, then __nonzero__ (if patch submitted) 3.x __bool__ first, then __nonzero__ The fewer variations there are in the algorithm, the better. -Fred -- Fred Drake ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
So just to clarify: 2.5 __nonzero__ only 2.6 __nonzero__ first, then __bool__ (if patch submitted) 3.x __bool__ first, then __nonzero__ Is this correct? On 10/29/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > 2007/10/28, Brett Cannon <[EMAIL PROTECTED]>: > > On 10/28/07, James Thiele <[EMAIL PROTECTED]> wrote: > > > PEP 361 lists __bool__ support as being possible for 2.6 backporting. > > > As of today the trunk build uses __nonzero__ like 2.5 but 3.0 alpha > > > uses __bool__. Has a decision been made on whether this will make the > > > cut for 2.6? > > > > > > In a more general vein, is there a cutoff date for producing a list of > > > 3.0 features which will be backported to 2.6? > > > > Backporting decisions have not been made as the feature set of 3.0 is > > still a moving target. Once we nail down the features (I am going to > > guess not until b1 at the earliest) then backporting will probably > > start. > > In this case, like many, the backport can't be an exact copy of the > 3.0 code: 2.6 *must* support __nonzero__. But it should also support > __bool__ as a fallback. I think it would be great if someone submited > a patch to implement this (though it isn't necessarily the highest > backporting priority). > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
I think several postings have explained better than I have on why __cmp__ is still very valuable. (See below.) Guido van Rossum posted earlier that he was willing to entertain a PEP to restore __cmp__, so I've attempted to create a draft PEP, posted here: http://www.dwheeler.com/misc/pep-cmp.txt Please let me know if it makes sense. Thanks. Greg Ewing stated "Why not provide a __richcmp__ method that directly connects with the corresponding type slot? All the comparisons eventually end up there anyway, so it seems like the right place to provide a one-stop comparison method in the 3.0 age." It _seems_ to me that this is the same as "__cmp__", and if so, let's just keep using the same name (there's nothing wrong with the name!). But maybe I just don't understand the comment, so explanation welcome. --- David A. Wheeler Aahz: >From my perspective, the real use case for cmp() is when you want to do >a three-way comparison of a "large" object (for example, a Decimal >instance). You can store the result of cmp() and then do a separate >three-way branch. and reply to the note "I'm having troubles coming up with things where the *basic* operator is really a cmp-like function.", there were two replies.. Guido van Rossum: >Here's one. When implementing the '<' operator on lists or tuples, you > really want to call the 'cmp' operator on the individual items, > because otherwise (if all you have is == and <) the algorithm becomes > something like "compare for equality until you've found the first pair > of items that are unequal; then compare those items again using < to > decide the final outcome". If you don't believe this, try to implement > this operation using only == or < without comparing any two items more > than once. and Greg Ewing: > Think of things like comparing a tuple. You need to work your > way along and recursively compare the elements. The decision > about when to stop always involves ==, whatever comparison > you're trying to do. So if e.g. you're doing <, then you have > to test each element first for <, and if that's false, test > it for ==. If the element is itself a tuple, it's doing this > on its elements too, etc., and things get very inefficient. > > If you have a single cmp operation that you can apply to the > elements, you only need to do it once for each element and it > gives you all the information you need. ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
2007/10/29, James Thiele <[EMAIL PROTECTED]>: > So just to clarify: > 2.5 __nonzero__ only > 2.6 __nonzero__ first, then __bool__ (if patch submitted) > 3.x __bool__ first, then __nonzero__ > > Is this correct? No. 3.x tests __bool__ only. > On 10/29/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > 2007/10/28, Brett Cannon <[EMAIL PROTECTED]>: > > > On 10/28/07, James Thiele <[EMAIL PROTECTED]> wrote: > > > > PEP 361 lists __bool__ support as being possible for 2.6 backporting. > > > > As of today the trunk build uses __nonzero__ like 2.5 but 3.0 alpha > > > > uses __bool__. Has a decision been made on whether this will make the > > > > cut for 2.6? > > > > > > > > In a more general vein, is there a cutoff date for producing a list of > > > > 3.0 features which will be backported to 2.6? > > > > > > Backporting decisions have not been made as the feature set of 3.0 is > > > still a moving target. Once we nail down the features (I am going to > > > guess not until b1 at the earliest) then backporting will probably > > > start. > > > > In this case, like many, the backport can't be an exact copy of the > > 3.0 code: 2.6 *must* support __nonzero__. But it should also support > > __bool__ as a fallback. I think it would be great if someone submited > > a patch to implement this (though it isn't necessarily the highest > > backporting priority). > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
On Oct 29, 2007, at 2:51 PM, Guido van Rossum wrote: > Makes sense, if you change the 3.x rule to > > 3.x __bool__ only. Even better! I think I'm going to like 3.0 if I ever get a chance to use it. ;-) -Fred -- Fred Drake ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] bug in i/o module buffering?
> I'm asking about b because the error message "TypeError: 'slice' > object does not support item deletion" would suggest that b is a slice > object. I agree that doesn't sound very likely given the code > though... :-( Could you step through this using pdb and investigate > some more? Perhaps there's a refcount error somewhere in the C code? I'll see if I can unfix my test code to reproduce the failure :-). Bill ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
> The SocketCloser class wasn't written with > the SSL use case in mind. I don't think it's just SSL. The problem is that it explicitly counts calls to "close()". So if you let the GC sweep up after you, that close() just doesn't get called, the circular refs persist, and the resource doesn't get collected till the backup GC runs (if it does). Waiting for that to happen, you might run out of a scarce system resource (file descriptors). A nasty timing-dependent bug, there. Hmmm, does real_close even get called in that case? In the C module, perhaps? > If you wanted to reuse the _socket after closing > the SSL instance, you'd have to wrap it in a fresh socket instance. > > Does that make sense? (Please do note the difference throughout > between _socket and socket, the former being defined in socketmodule.c > and the latter in socket.py.) That's what I do with SSLSocket, pretty much. I worry that doing it with socket.socket might break a lot of non-TCP code, though. And perhaps it's overkill. Why not move the count of how many SocketIO instances are pointing to it into the socket.socket class again, as it was in 2.x? I don't think you're gaining anything with the circular data structure of SocketCloser. Add a "_closed" property, and "__del__" method to socket.socket (which calls "close()"). Remove SocketCloser. You're finished, and there's one less class to maintain. And, ref your other comments, why not call SocketIO "TCPStream"? It would make things much clearer. Also, is it too late to rename socket.socket to "socket.Socket"? There are only a handful of references to "socket.socket" outside of the socket and ssl modules. Bill ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
On 10/29/07, David A. Wheeler <[EMAIL PROTECTED]> wrote: > I think several postings have explained better than I have on why __cmp__ is > still very valuable. (See below.) > > Guido van Rossum posted earlier that he was willing to entertain a PEP to > restore __cmp__, so I've attempted to create a draft PEP, posted here: > http://www.dwheeler.com/misc/pep-cmp.txt > Please let me know if it makes sense. Thanks. > > Greg Ewing stated "Why not provide a __richcmp__ method that directly connects > with the corresponding type slot? All the comparisons eventually end up there > anyway, so it seems like the right place to provide a one-stop comparison > method in the 3.0 age." > It _seems_ to me that this is the same as "__cmp__", and if so, let's just > keep using the same name (there's nothing wrong with the name!). But maybe I > just don't understand the comment, so explanation welcome. I believe the intent was for __richcmp__ to take an argument indicating what sort of comparison is to be done (as tp_richcompare does in C.) ie, you'd write code like this: def __richcmp__(self, other, op): if !isinstance(other, MyType): return NotImplemented return richcmp(self.foo, other.foo, op) Short-circuiting of equality checks (due to identity or interning) would work right. Likewise, there's no odd behaviour with comparable-but-unorderable types. It's not clear to me how many distinct operations you'd need though, or how acceptable reflections would be. Would only two operations, equality and ordering, be sufficient? Just what are the non-symmetric use cases the current design caters to? > > > > --- David A. Wheeler > > > > Aahz: > >From my perspective, the real use case for cmp() is when you want to do > >a three-way comparison of a "large" object (for example, a Decimal > >instance). You can store the result of cmp() and then do a separate > >three-way branch. > > and reply to the note "I'm having troubles coming up with things where > the *basic* operator is really a cmp-like function.", there were two replies.. > > > Guido van Rossum: > >Here's one. When implementing the '<' operator on lists or tuples, you > > really want to call the 'cmp' operator on the individual items, > > because otherwise (if all you have is == and <) the algorithm becomes > > something like "compare for equality until you've found the first pair > > of items that are unequal; then compare those items again using < to > > decide the final outcome". If you don't believe this, try to implement > > this operation using only == or < without comparing any two items more > > than once. > > and > > Greg Ewing: > > Think of things like comparing a tuple. You need to work your > > way along and recursively compare the elements. The decision > > about when to stop always involves ==, whatever comparison > > you're trying to do. So if e.g. you're doing <, then you have > > to test each element first for <, and if that's false, test > > it for ==. If the element is itself a tuple, it's doing this > > on its elements too, etc., and things get very inefficient. > > > > If you have a single cmp operation that you can apply to the > > elements, you only need to do it once for each element and it > > gives you all the information you need. > > ___ > Python-3000 mailing list > Python-3000@python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/rhamph%40gmail.com > -- Adam Olsen, aka Rhamphoryncus ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
> Actually, I'm still up for tweaks to the I/O model if it solves a real
> problem, as long as most of the high-level APIs stay the same (there
> simply is too much code that expects those to behave a certain way).
>
> I don't quite understand what you mean by inverted though.
I'm actually thinking more in terms of avoiding future problems. I
thought we'd discussed this a few months ago, but here it is again:
I'd break up the BaseIO class into a small set of base classes, so that
we can be more explicit about what a particular kind of I/O channel is
or is not:
(Please excuse typos, I'm generating this off-the-cuff -- does
@abstract actually exist?)
-
class IOStream:
@abstract
def close(self):
@property
def closed(self):
class InputIOStream (IOStream):
@abstract
def read(self, buffer=None, nbytes=None):
class OutputIOStream (IOStream):
@abstract
def write(self, bytes):
@abstract
def flush(self):
class SeekableIOStream (IOStream):
@abstract
def tell(self):
@abstract
def seek(self):
@abstract
def truncate(self):
class SystemIOStream (IOStream):
@property
def fileno(self):
@property
def isatty (self):
class TextInputStream (InputIOStream):
@abstract
def readline(self):
@abstract
def readlines(self):
class TextOutputStream (InputIOStream):
@abstract
def readline(self):
@abstract
def readlines(self):
class FileStream (SystemIOStream, SeekableIOStream):
@property
name
@property
mode
# note that open() would return FileStream mixed with one or both of
# {Text}InputStream and {Text}OutputStream, depending on the "mode".
class StringIO (SeekableIOStream):
# again, mixed with IO modes, depending on "mode".
-
I think of this as inverted, because it puts primitives like "read"
and "write" at the lowest layers, not above things like "fileno" or
"truncate", which are very specialized and should only apply to a
subset of I/O channels.
I realize that there are some practical problems with this; such as
making _fileio.FileIO inherit from (multiple) Python base classes.
Bill
___
Python-3000 mailing list
Python-3000@python.org
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
2007/10/29, Bill Janssen <[EMAIL PROTECTED]>:
> > Actually, I'm still up for tweaks to the I/O model if it solves a real
> > problem, as long as most of the high-level APIs stay the same (there
> > simply is too much code that expects those to behave a certain way).
> >
> > I don't quite understand what you mean by inverted though.
>
> I'm actually thinking more in terms of avoiding future problems.
Can you remind me of what future problems again?
> I thought we'd discussed this a few months ago, but here it is again:
>
> I'd break up the BaseIO class into a small set of base classes, so that
> we can be more explicit about what a particular kind of I/O channel is
> or is not:
I see, static type checks in favor of dynamic behavior checks -- e.g.
isinstance(s, SeekableIOStream) rather than s.seekable(). If that's
all, I guess I already expressed earlier I don't really like that --
in practice I think the dynamic checks are more flexible, and the
class hierarchy you're proposing is hard to implement in C (where
unfortunately I'm restricted to single inheritance). E.g. depending on
how a program is invoked, sys.stdin may be seekable or it may not be.
--Guido
> (Please excuse typos, I'm generating this off-the-cuff -- does
> @abstract actually exist?)
>
> -
>
> class IOStream:
>
>@abstract
>def close(self):
>
>@property
>def closed(self):
>
> class InputIOStream (IOStream):
>
>@abstract
>def read(self, buffer=None, nbytes=None):
>
> class OutputIOStream (IOStream):
>
>@abstract
>def write(self, bytes):
>
>@abstract
>def flush(self):
>
> class SeekableIOStream (IOStream):
>
>@abstract
>def tell(self):
>
>@abstract
>def seek(self):
>
>@abstract
>def truncate(self):
>
> class SystemIOStream (IOStream):
>
>@property
>def fileno(self):
>
>@property
>def isatty (self):
>
> class TextInputStream (InputIOStream):
>
>@abstract
>def readline(self):
>
>@abstract
>def readlines(self):
>
> class TextOutputStream (InputIOStream):
>
>@abstract
>def readline(self):
>
>@abstract
>def readlines(self):
>
> class FileStream (SystemIOStream, SeekableIOStream):
>
>@property
>name
>
>@property
>mode
>
> # note that open() would return FileStream mixed with one or both of
> # {Text}InputStream and {Text}OutputStream, depending on the "mode".
>
> class StringIO (SeekableIOStream):
>
> # again, mixed with IO modes, depending on "mode".
>
> -
>
> I think of this as inverted, because it puts primitives like "read"
> and "write" at the lowest layers, not above things like "fileno" or
> "truncate", which are very specialized and should only apply to a
> subset of I/O channels.
>
> I realize that there are some practical problems with this; such as
> making _fileio.FileIO inherit from (multiple) Python base classes.
>
> Bill
>
>
>
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-3000 mailing list
Python-3000@python.org
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
2007/10/29, Bill Janssen <[EMAIL PROTECTED]>: > > The SocketCloser class wasn't written with > > the SSL use case in mind. > > I don't think it's just SSL. The problem is that it explicitly counts > calls to "close()". So if you let the GC sweep up after you, that > close() just doesn't get called, the circular refs persist, and the > resource doesn't get collected till the backup GC runs (if it does). > Waiting for that to happen, you might run out of a scarce system > resource (file descriptors). A nasty timing-dependent bug, there. Ouch. Unfortunately adding a __del__() method that calls close() won't really help, as the cyclic GC refuses to do anything with objects having a __del__. This needs more thinking than I have time for right now, but i agree we need to fix it. > Hmmm, does real_close even get called in that case? In the C module, > perhaps? The C module will certainly close the fd when the object goes away. The question is, is that soon enough. > > If you wanted to reuse the _socket after closing > > the SSL instance, you'd have to wrap it in a fresh socket instance. > > > > Does that make sense? (Please do note the difference throughout > > between _socket and socket, the former being defined in socketmodule.c > > and the latter in socket.py.) > > That's what I do with SSLSocket, pretty much. I worry that doing it > with socket.socket might break a lot of non-TCP code, though. And > perhaps it's overkill. > > Why not move the count of how many SocketIO instances are pointing to > it into the socket.socket class again, as it was in 2.x? I don't > think you're gaining anything with the circular data structure of > SocketCloser. Add a "_closed" property, and "__del__" method to > socket.socket (which calls "close()"). Remove SocketCloser. You're > finished, and there's one less class to maintain. I'll look into this later. > And, ref your other comments, why not call SocketIO "TCPStream"? It > would make things much clearer. Good idea; SocketIO made more sense when it was part of io.py. > Also, is it too late to rename socket.socket to "socket.Socket"? > There are only a handful of references to "socket.socket" outside of > the socket and ssl modules. Really? AFAIK everyone who opens a socket calls it. I'd be okay with calling the class Socket and having a factory function named socket though. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
I'm a bit too busy to look into this right now; I hope one or two more rounds of feedback on the PEP will get it into a state where I can review it more easily. Having a patch to go with it would be immensely helpful as well (in fact I'd say that without a patch it's unlikely to happen). --Guido 2007/10/29, David A. Wheeler <[EMAIL PROTECTED]>: > I think several postings have explained better than I have on why __cmp__ is > still very valuable. (See below.) > > Guido van Rossum posted earlier that he was willing to entertain a PEP to > restore __cmp__, so I've attempted to create a draft PEP, posted here: > http://www.dwheeler.com/misc/pep-cmp.txt > Please let me know if it makes sense. Thanks. > > Greg Ewing stated "Why not provide a __richcmp__ method that directly connects > with the corresponding type slot? All the comparisons eventually end up there > anyway, so it seems like the right place to provide a one-stop comparison > method in the 3.0 age." > It _seems_ to me that this is the same as "__cmp__", and if so, let's just > keep using the same name (there's nothing wrong with the name!). But maybe I > just don't understand the comment, so explanation welcome. > > > > --- David A. Wheeler > > > > Aahz: > >From my perspective, the real use case for cmp() is when you want to do > >a three-way comparison of a "large" object (for example, a Decimal > >instance). You can store the result of cmp() and then do a separate > >three-way branch. > > and reply to the note "I'm having troubles coming up with things where > the *basic* operator is really a cmp-like function.", there were two replies.. > > > Guido van Rossum: > >Here's one. When implementing the '<' operator on lists or tuples, you > > really want to call the 'cmp' operator on the individual items, > > because otherwise (if all you have is == and <) the algorithm becomes > > something like "compare for equality until you've found the first pair > > of items that are unequal; then compare those items again using < to > > decide the final outcome". If you don't believe this, try to implement > > this operation using only == or < without comparing any two items more > > than once. > > and > > Greg Ewing: > > Think of things like comparing a tuple. You need to work your > > way along and recursively compare the elements. The decision > > about when to stop always involves ==, whatever comparison > > you're trying to do. So if e.g. you're doing <, then you have > > to test each element first for <, and if that's false, test > > it for ==. If the element is itself a tuple, it's doing this > > on its elements too, etc., and things get very inefficient. > > > > If you have a single cmp operation that you can apply to the > > elements, you only need to do it once for each element and it > > gives you all the information you need. > > ___ > Python-3000 mailing list > Python-3000@python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
On 10/29/07, David A. Wheeler <[EMAIL PROTECTED]> wrote: > I think several postings have explained better than I have on why __cmp__ is > still very valuable. (See below.) > > Guido van Rossum posted earlier that he was willing to entertain a PEP to > restore __cmp__, so I've attempted to create a draft PEP, posted here: > http://www.dwheeler.com/misc/pep-cmp.txt > Please let me know if it makes sense. Thanks. I think the PEP's a little misleading in that it makes it sound like defining __lt__, __gt__, etc. is inefficient. I think you want to be explicit about where __lt__, __gt__ are efficient, and where __cmp__ is efficient. For example:: * __lt__ is more efficient for sorting (given the current implementation) * __cmp__ is more efficient for comparing sequences like tuples, where you always need to check for equality first, and you don't want to have to do an == check followed by a < check if you can do them both at the same time. (This is basically the same argument as for Decimal -- why do two comparisons when you can do one?) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
> Really? AFAIK everyone who opens a socket calls it. Sorry, I meant only a handful (10?) in the standard library. > I'd be okay with calling the class Socket and having a factory > function named socket though. Ah, good idea. Bill ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] 3K bytes I/O?
And for non-unicode inputs the code should use the PEP 3118 buffer API rather than PyBytes_ or PyString_ or whatnot. On 10/26/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > 2007/10/26, Bill Janssen <[EMAIL PROTECTED]>: > > I'm looking at the Py3K SSL code, and have a question: > > > > What's the upshot of the bytes/string decisions in the C world? Is > > PyString_* now all about immutable bytes, and PyUnicode_* about > > strings? There still seem to be a lot of encode/decode methods in > > stringobject.h, operations which I'd expect to be in unicodeobject.h. > > I think the PyString encode/decode APIs should all go; use the > corresponding PyUnicode ones. > > I recommend that you write your code to assume PyBytes for > encoded/binary data, and PyUnicode for text; at some point we'll > substitute PyString for most cases where PyBytes is currently used: > that will happen once PyString is called bytes in at the Python level, > and PyBytes will be called buffer. But that's still a while off. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > ___ > Python-3000 mailing list > Python-3000@python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/greg%40krypto.org > ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
David A. Wheeler wrote: > Greg Ewing stated "Why not provide a __richcmp__ method that directly connects > with the corresponding type slot? > It _seems_ to me that this is the same as "__cmp__", No, it's not -- a __richcmp__ method would take an extra argument specifying which of the six comparison operations to perform, and return a boolean instead of -1, 0, 1. Giving it the same name as the old __cmp__ would be confusing, I think. -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] __bool__ in 2.6?
James Thiele wrote: > 3.x __bool__ first, then __nonzero__ Does 3.x need __nonzero__ at all? -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
2007/10/29, Greg Ewing <[EMAIL PROTECTED]>: > I wrote: > > > Seems to me that a socket should already *be* a file, > > so it shouldn't need a makefile() method and you > > shouldn't have to mess around with filenos. > > Guido van Rossum wrote: > > > That model fits TCP/IP streams just fine, but doesn't work so well for > > UDP and other odd socket types. > > No, but I think that a socket should have read() and > write() methods that work if it happens to be a socket > of an appropriate kind. Unix lets you use read and write > as synonyms for send and recv on stream sockets, and > it's surprising that Python doesn't do the same. That's because I don't find the synonyms a good idea. > At the very least, it should be possible to wrap > any of the higher-level I/O stack objects around a > stream socket directly. Why? What problem does this solve? > > The real issue seems to be file descriptor GC. Maybe we haven't > > written down the rules clearly enough for when the fd is supposed to > > be GC'ed > > I don't see what's so difficult about this. Each file > descriptor should be owned by exactly one object. If > two objects need to share a fd, then you dup() it so > that each one has its own fd. When the object is > close()d or GCed, it closes its fd. On Windows you can't dup() a fd. > However, I don't see that it should be necessary for > objects to share fds in the first place. Buffering > layers should wrap directly around the the object > being buffered, whether a file or socket or something > else. Then whether the socket has a fd or not is > an implementation detail of the socket object, so > there's no problem on Windows. There's a tension though between using GC and explicit closing. A fairly nice model would be that the lowest-level object "owns" the fd and is the one to close it when it is GC'ed. However for various reasons we don't want to rely on GC to close fds, since that may delay closing in Jython and when there happens to be an innocent reference keeping the lowest-level socket object alive (e.g. someone still has it in their stack frame or traceback). So we end up having to implement a second reference counting scheme on top of close() calls. Which is what we did. But now just dropping the last reference to an object doesn't call close(), so explicit closes suddenly become mandatory instead of recommended good practice. Adding __del__ as an alias for close might help, except this makes circular references a primary sin (since the cycle GC doesn't like calling __del__). I guess there really is no way around this solution though, and we'll just have to make extra sure not to create cycles during normal usage patterns, or use weak references in those cases where we can't avoid them. I think this is the way to go, together with changing the Socket class from subclassing _socket to wrapping one. --Guido > Bill Janssen wrote: > > > Back to your initial mail (which is > > more relevant than Greg Ewing's snipe!): > > What snipe? I'm trying to make a constructive suggestion. > > > then in some > > cases *closes* the socket (thereby reasonably rendering the socket > > *dead*), *then* returns the "file" to the caller as part of the > > response. > > I don't understand that. What good can returning a *closed* file > object possibly do anyone? > > -- > Greg > ___ > Python-3000 mailing list > Python-3000@python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
I wrote: > Seems to me that a socket should already *be* a file, > so it shouldn't need a makefile() method and you > shouldn't have to mess around with filenos. Guido van Rossum wrote: > That model fits TCP/IP streams just fine, but doesn't work so well for > UDP and other odd socket types. No, but I think that a socket should have read() and write() methods that work if it happens to be a socket of an appropriate kind. Unix lets you use read and write as synonyms for send and recv on stream sockets, and it's surprising that Python doesn't do the same. At the very least, it should be possible to wrap any of the higher-level I/O stack objects around a stream socket directly. > The real issue seems to be file descriptor GC. Maybe we haven't > written down the rules clearly enough for when the fd is supposed to > be GC'ed I don't see what's so difficult about this. Each file descriptor should be owned by exactly one object. If two objects need to share a fd, then you dup() it so that each one has its own fd. When the object is close()d or GCed, it closes its fd. However, I don't see that it should be necessary for objects to share fds in the first place. Buffering layers should wrap directly around the the object being buffered, whether a file or socket or something else. Then whether the socket has a fd or not is an implementation detail of the socket object, so there's no problem on Windows. Bill Janssen wrote: > Back to your initial mail (which is > more relevant than Greg Ewing's snipe!): What snipe? I'm trying to make a constructive suggestion. > then in some > cases *closes* the socket (thereby reasonably rendering the socket > *dead*), *then* returns the "file" to the caller as part of the > response. I don't understand that. What good can returning a *closed* file object possibly do anyone? -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
2007/10/29, Greg Ewing <[EMAIL PROTECTED]>: > David A. Wheeler wrote: > > Greg Ewing stated "Why not provide a __richcmp__ method that > > directly connects with the corresponding type slot? > > > It _seems_ to me that this is the same as "__cmp__", > > No, it's not -- a __richcmp__ method would take an extra > argument specifying which of the six comparison operations > to perform, and return a boolean instead of -1, 0, 1. Eh? Shouldn't it return True, False or NotImplemented if that's the interface? > Giving it the same name as the old __cmp__ would be > confusing, I think. For sure. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
Adam Olsen wrote: > It's not clear to me how many distinct operations you'd need though, > or how acceptable reflections would be. My intention was just to directly expose the tp_richcmp slot, so there would be six. To make things easier in the common case, there could perhaps be a utility function that would take a comparison operation code and a -1, 0, 1 value and return the appropriate boolean. Then a __richcmp__ method could be written very similarly to the way a __cmp__ method is now. It might even be possible for 2to3 to convert __cmp__ methods to __richcmp__ methods automatically. -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
Guido van Rossum wrote: > having read() > and write() methods on the _socket object is not the solution. It's not a necessary part of the solution, I agree. I just don't see what purpose is served by requiring an extra layer of wrapper between a socket and the other I/O layers. That's not a necessary part of the solution either. -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
Guido van Rossum wrote: > That's because I don't find the synonyms a good idea. Even if it means that stream sockets then have the same interface as all other stream-like objects in the I/O system, so buffering layers can be used on them, etc.? That seems like a rather good reason to me. If you want to be pedantic about not having synonyms, then fix send() and recv() so that they only work on *non*-stream sockets, or have different classes for stream and non-stream sockets. In other words, to my mind, for stream sockets it's send and recv that are synonyms for read and write, not the other way around. > On Windows you can't dup() a fd. Oh, blarg. Forget that part, then. But I still think it shouldn't be necessary to share fds between different objects in the first place. This is the problem that would be solved by making sockets have an interface that is directly usable by higher layers of the I/O system. There would be no need to reach down below the socket object and grab its fd, so the socket would have complete ownership of it, and it would get closed when the socket object eventually went away. This would happen at the C level, so cycles and __del__ methods wouldn't be a serious problem. -- Greg ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] socket GC worries
2007/10/29, Greg Ewing <[EMAIL PROTECTED]>: > Guido van Rossum wrote: > > > That's because I don't find the synonyms a good idea. > > Even if it means that stream sockets then have the > same interface as all other stream-like objects in > the I/O system, so buffering layers can be used on > them, etc.? That seems like a rather good reason to > me. > > If you want to be pedantic about not having synonyms, > then fix send() and recv() so that they only work > on *non*-stream sockets, or have different classes > for stream and non-stream sockets. > > In other words, to my mind, for stream sockets it's > send and recv that are synonyms for read and write, > not the other way around. > > > On Windows you can't dup() a fd. > > Oh, blarg. Forget that part, then. > > But I still think it shouldn't be necessary to share > fds between different objects in the first place. > > This is the problem that would be solved by making > sockets have an interface that is directly usable by > higher layers of the I/O system. There would be no > need to reach down below the socket object and grab > its fd, so the socket would have complete ownership > of it, and it would get closed when the socket > object eventually went away. This would happen at > the C level, so cycles and __del__ methods wouldn't > be a serious problem. Having the SocketIO wrapper works just as well. I agree we need some refactoring to deal with the ownership issue better, but having read() and write() methods on the _socket object is not the solution. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
On 10/29/07, Greg Ewing <[EMAIL PROTECTED]> wrote: > Adam Olsen wrote: > > It's not clear to me how many distinct operations you'd need though, > > or how acceptable reflections would be. > > My intention was just to directly expose the tp_richcmp > slot, so there would be six. > > To make things easier in the common case, there could > perhaps be a utility function that would take a comparison > operation code and a -1, 0, 1 value and return the > appropriate boolean. Then a __richcmp__ method could be > written very similarly to the way a __cmp__ method is > now. It might even be possible for 2to3 to convert > __cmp__ methods to __richcmp__ methods automatically. It'd be simpler still if we only had __cmp__ and __eq__. I just don't understand the use cases where that's not sufficient. Hrm. I guess set's subset checking requires more relationships than __cmp__ provides. Abandoning that feature probably isn't an option, so nevermind me. (Although, if we really wanted we could use -2/+2 to mean subset/superset, while -1/+1 mean smaller/larger.) -- Adam Olsen, aka Rhamphoryncus ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Please re-add __cmp__ to python 3000
On 10/29/07, Steven Bethard <[EMAIL PROTECTED]> wrote: > On 10/29/07, David A. Wheeler <[EMAIL PROTECTED]> wrote: > > I think several postings have explained better than I have on why __cmp__ > > is still very valuable. (See below.) > > > > Guido van Rossum posted earlier that he was willing to entertain a PEP to > > restore __cmp__, so I've attempted to create a draft PEP, posted here: > > http://www.dwheeler.com/misc/pep-cmp.txt > > Please let me know if it makes sense. Thanks. > > I think the PEP's a little misleading in that it makes it sound like > defining __lt__, __gt__, etc. is inefficient. I think you want to be > explicit about where __lt__, __gt__ are efficient, and where __cmp__ > is efficient. For example:: > > * __lt__ is more efficient for sorting (given the current implementation) > * __cmp__ is more efficient for comparing sequences like tuples, where > you always need to check for equality first, and you don't want to > have to do an == check followed by a < check if you can do them both > at the same time. (This is basically the same argument as for Decimal > -- why do two comparisons when you can do one?) When implementing a large, totally ordered object (>=2 fields), both __lt__ and __cmp__ should probably be implemented by calling __cmp__ on the fields. If you decide to implement __lt__ by letting it forward to __cmp__, the cutoff might be at 3 fields. Partial orders (what the PEP calls "asymmetric classes") cannot, of course, be implemented with __cmp__ and should have it return NotImplemented. Well, if we wanted to diverge from most other languages, we could extend __cmp__ to let it return a distinguished "Unordered" value, which returns false on all comparisons with 0. This is similar to Fortress's approach, which returns one of 4 values from a PartialOrder's CMP operator: EqualTo, LessThan, GreaterThan, and Unordered. Haskell has only a total ordering class in the core libraries, while Scala has a PartiallyOrdered trait that returns None from its compare method for unordered values. For Python, I think I favor reviving __cmp__ for totally ordered types, and asking that partially ordered ones return NotImplemented from it explicitly. Jeffrey ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
