Re: [Python-Dev] should list's call to __len__ swallow SystemExit?
Dino Viehland wrote: We had a bug reported that effectively boils down to we’re not swallowing exceptions when list calls __len__ (http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=20598). We can obviously make the change to catch exceptions here in IronPython even if it seems like a bad idea to me ☺ But CPython seems to catch not only normal exceptions, but also SystemExit. It seems like there’s been a move away from this so I thought I’d mention it here. I tested it on 2.6.1 and 3.0. I'd agree that CPython appears to be the one misbehaving here, rather than IronPython. Opening a new issue at bugs.python.org would be the best way forward. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] socket.create_connection slow
Greetings. I spent the morning trying to find out why the disabled tests in test_xmlrpc.py ran so slowly on my vista box. After much digging, I found that it boiled down to socket.create_connection() trying to connect to localhost, port. You see, it does a getaddrinfo() and then tries to connect using all the various addresses it finds until it succeeds. On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? I can think of two things to make this better: 1) Make sure that AF_INET addresses are tried first in socket.create_connection() 2) Have the SocketServer create a listening socket for each address family by default. Any thoughts? K ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Kristján Valur Jónsson kristjan at ccpgames.com writes: On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? Is the fix ok? What if the user wanted to connect to an XMLRPC server using IPv6? 2) Have the SocketServer create a listening socket for each address family by default. I don't see how that fixes the root of the problem, not all XMLRPC servers are implemented using SocketServer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] should list's call to __len__ swallow SystemExit?
On Wed, Jan 14, 2009 at 6:12 AM, Nick Coghlan ncogh...@gmail.com wrote: Dino Viehland wrote: We had a bug reported that effectively boils down to we're not swallowing exceptions when list calls __len__ ( http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=20598 ). We can obviously make the change to catch exceptions here in IronPython even if it seems like a bad idea to me ☺ But CPython seems to catch not only normal exceptions, but also SystemExit. It seems like there's been a move away from this so I thought I'd mention it here. I tested it on 2.6.1 and 3.0. I'd agree that CPython appears to be the one misbehaving here, rather than IronPython. Opening a new issue at bugs.python.org would be the best way forward. There is already a bug for this, I believe: http://bugs.python.org/issue1242657 Karen ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
The fix is for the unittest only, where a server is started on a separate thread using Ipv4 on localhost. Now, the problem I was describing in the mail isn't specific to xmlrpc, but for Anyone that wants to use socket.create_connection() to create a stream socket to a host whose name has both an Ipv4 and Ipv6 address. Unless the host is listening on its Ipv6 port, an unsuccessful connection attempt will first be made, taking approximately one second. Kristján -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Antoine Pitrou Sent: 14. janúar 2009 11:34 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Kristján Valur Jónsson kristjan at ccpgames.com writes: On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? Is the fix ok? What if the user wanted to connect to an XMLRPC server using IPv6? 2) Have the SocketServer create a listening socket for each address family by default. I don't see how that fixes the root of the problem, not all XMLRPC servers are implemented using SocketServer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Availability of IPv6 support in the socket module
Are there any current plans for 2.7/3.1 to have inet_pton() and inet_ntop() made available via the socket module? Not sure how feasible or difficult doing this would be across all support Python platforms but it would certainly be useful, especially now there is talk about adding IPv4/IPv6 address manipulation support to the standard library (http://bugs.python.org/issue3959). This would provide handy C-level speed ups to IPv6 operations on interpreters where the socket module is available. Google App Engine's Python interpreter is one place where I believe inet_ntoa/aton and possibly inet_ntop/pton are not and will not be made available (by design). In such cases, a library could easily resort to ubiquitous (slightly slower) pure-Python fallbacks. Also, does anyone have a feeling for how available thet AF_INET6 constant is is across all Python's many supported platforms? Thanks, David Moss ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Victor Stinner wrote: Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. This is something of a bugbear on Vista in general. Doing local web-development with localhost can be really painful until you realise that switching to 127.0.0.1 solves the problem... Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
I have no idea why the connect refusal takes so long. Maybe it's a vista thing? from socket import * socket(AF_INET6).connect((::1, 8080)) takes about one second to report active refusal. But so does an IPv4 connect. Maybe it is some kind of DOS attack throttling? I couldn't find any info. I've already asked the client in the test to use IPV4 by specifying the connection address as an IPv4 tuple (http://127.0.0.1:...;). I see no other way to do it without extensive subclassing because the HTTPConnection() class uses socket.create_connection(). Cheers, Kristján -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Victor Stinner Sent: 14. janúar 2009 12:46 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
This may be way out on a limb, but could it be a reverse lookup issue? -Original Message- From: python-dev-bounces+billy.earney=gmail@python.org [mailto:python-dev-bounces+billy.earney=gmail@python.org] On Behalf Of Kristján Valur Jónsson Sent: Wednesday, January 14, 2009 8:36 AM To: Victor Stinner; python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow I have no idea why the connect refusal takes so long. Maybe it's a vista thing? from socket import * socket(AF_INET6).connect((::1, 8080)) takes about one second to report active refusal. But so does an IPv4 connect. Maybe it is some kind of DOS attack throttling? I couldn't find any info. I've already asked the client in the test to use IPV4 by specifying the connection address as an IPv4 tuple (http://127.0.0.1:...;). I see no other way to do it without extensive subclassing because the HTTPConnection() class uses socket.create_connection(). Cheers, Kristján -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Victor Stinner Sent: 14. janúar 2009 12:46 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/billy.earney%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Availability of IPv6 support in the socket module
2009/1/14 DrKJam drk...@gmail.com Are there any current plans for 2.7/3.1 to have inet_pton() and inet_ntop() made available via the socket module? Not sure how feasible or difficult doing this would be across all support Python platforms but it would certainly be useful, especially now there is talk about adding IPv4/IPv6 address manipulation support to the standard library (http://bugs.python.org/issue3959). This would provide handy C-level speed ups to IPv6 operations on interpreters where the socket module is available. Google App Engine's Python interpreter is one place where I believe inet_ntoa/aton and possibly inet_ntop/pton are not and will not be made available (by design). In such cases, a library could easily resort to ubiquitous (slightly slower) pure-Python fallbacks. Also, does anyone have a feeling for how available thet AF_INET6 constant is is across all Python's many supported platforms? Thanks, David Moss A real RTFM moment. I'm using Windows and I see Linux has had this support since 2.3 or earlier. Please ignore. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Michael Foord wrote: Victor Stinner wrote: Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. This is something of a bugbear on Vista in general. Doing local web-development with localhost can be really painful until you realise that switching to 127.0.0.1 solves the problem... It barfs on Macs as well: indeed, it is worse, because the connection just fails there, rather than trying IPv6 and then falling back to IPv4. For instance, tunneling a connection over SSH to a Mac box via '-L :localhost:' will fail to connect at all, unless the server is listening on IPv6. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJbgo9+gerLs4ltQ4RAvK9AKCfWhQx7ntw+sUNK7FCPU+Kb9jp5QCdEqCu 9BXvzTgBKipSCtA3SdydqjI= =tYDj -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Hardly. Successful connects complete in a jiffy, only actively refused ones take a second to do so. I suspect some michief in the vista tcp stack. -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Billy Earney Sent: 14. janúar 2009 15:43 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow This may be way out on a limb, but could it be a reverse lookup issue? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] should list's call to __len__ swallow SystemExit?
_PyObject_LengthHint() is specified to never fail.
If any exception occurs along the way, it returns a
default value. In the context of checking for length
hints from an iterator, this seems reasonable to me.
If you want this changed, I can use a negative return
value for other than an attribute error, and modify
the calling code to handle the exception.
To me this isn't worth making the code slower and
more complex. But I can also see wanting to catch
a SystemError at any possible step.
I presume this same issue occurs everywhere the C API
has a *this never fails* specification so that we have
simpler, faster calling code at the expense of being able
to raise a SystemError in every possible piece of code.
Raymond
From: Guido van Rossum gu...@python.org
There seems to be an unconditional PyErr_Clear() in
_PyObject_LengthHint(). I think that could and should be much more
careful; it probably should only ignore AttributeErrors (though there
may be unittests to the contrary).
On Tue, Jan 13, 2009 at 8:24 PM, Dino Viehland di...@microsoft.com wrote:
We had a bug reported that effectively boils down to we're not swallowing exceptions when list calls __len__
(http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=20598).
We can obviously make the change to catch exceptions here in IronPython even if it seems like a bad idea to me ☺ But CPython
seems to catch not only normal exceptions, but also SystemExit. It seems like there's been a move away from this so I thought
I'd mention it here. I tested it on 2.6.1 and 3.0.
import sys
class A(object):
def __iter__(self): return iter(range(10))
def __len__(self):
try:
print('exiting')
sys.exit(1)
except Exception as e:
print('can I catch it?', e)
list(A())
which prints:
exiting
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] should list's call to __len__ swallow SystemExit?
On Wed, Jan 14, 2009 at 10:11 AM, Raymond Hettinger pyt...@rcn.com wrote:
_PyObject_LengthHint() is specified to never fail.
If any exception occurs along the way, it returns a
default value. In the context of checking for length
hints from an iterator, this seems reasonable to me.
If you want this changed, I can use a negative return
value for other than an attribute error, and modify
the calling code to handle the exception.
To me this isn't worth making the code slower and
more complex. But I can also see wanting to catch
a SystemError at any possible step.
It has the potential of masking errors, and thus I'd like to see it fixed.
I presume this same issue occurs everywhere the C API
has a *this never fails* specification so that we have
simpler, faster calling code at the expense of being able
to raise a SystemError in every possible piece of code.
This never fails C APIs that invoke Python code (or e.g. allocate
memory) are not supposed to exist in CPython, for the reason above.
There used to be several but we gradually killed them all. I'm sorry I
wasn't involved more deeply in the review of this feature, I would
have warned about it.
Raymond
From: Guido van Rossum gu...@python.org
There seems to be an unconditional PyErr_Clear() in
_PyObject_LengthHint(). I think that could and should be much more
careful; it probably should only ignore AttributeErrors (though there
may be unittests to the contrary).
On Tue, Jan 13, 2009 at 8:24 PM, Dino Viehland di...@microsoft.com
wrote:
We had a bug reported that effectively boils down to we're not swallowing
exceptions when list calls __len__
(http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=20598).
We can obviously make the change to catch exceptions here in IronPython
even if it seems like a bad idea to me ☺ But CPython seems to catch not
only normal exceptions, but also SystemExit. It seems like there's been a
move away from this so I thought I'd mention it here. I tested it on 2.6.1
and 3.0.
import sys
class A(object):
def __iter__(self): return iter(range(10))
def __len__(self):
try:
print('exiting')
sys.exit(1)
except Exception as e:
print('can I catch it?', e)
list(A())
which prints:
exiting
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/python%40rcn.com
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Can you find out why it takes a second? That should not happen; it should fail immediately (assuming no server is listening). Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? Yes, but possibly with your operating system or installation. We need to understand what is happening first before trying to fix it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Anyone that wants to use socket.create_connection() to create a stream socket to a host whose name has both an Ipv4 and Ipv6 address. Unless the host is listening on its Ipv6 port, an unsuccessful connection attempt will first be made, taking approximately one second. Again, it is a bug in the system or the installation takes a second. There should be no timeout, but an immediate error. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
It barfs on Macs as well: indeed, it is worse, because the connection just fails there, rather than trying IPv6 and then falling back to IPv4. That depends on the application. Some applications fall back (as they should, if they added their support for IPv6 correctly), some don't. For instance, tunneling a connection over SSH to a Mac box via '-L :localhost:' will fail to connect at all, unless the server is listening on IPv6. That's actually bug in sshd. It should fall back to connecting through v4, but doesn't. It's not system specific; sshd as included in Debian has the same bug. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
I have no idea why the connect refusal takes so long. Can you run wireshark, to find out whether it's sending out any requests that don't get responses? Could it be that your firewall is discarding the connection request? (rather than sending an ICMP destination unreachable back) Anything added to the event log? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] should list's call to __len__ swallow SystemExit?
If you want this changed, I can use a negative return value for other than an attribute error, and modify the calling code to handle the exception. To me this isn't worth making the code slower and more complex. But I can also see wanting to catch a SystemError at any possible step. It has the potential of masking errors, and thus I'd like to see it fixed. No problem. I'll take care of this one. Since it's an internal API, it should be easy. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] should list's call to __len__ swallow SystemExit?
Karen Tracey wrote: There is already a bug for this, I believe: http://bugs.python.org/issue1242657 Thanks. Raymond, based on your other message, I kicked that issue in your direction. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r68547 - in python/trunk/Lib/test: test_datetime.py test_os.py
On Mon, 12 Jan 2009 19:09:28 +0100 (CET), kristjan.jonsson python-check...@python.org wrote: Author: kristjan.jonsson Date: Mon Jan 12 19:09:27 2009 New Revision: 68547 Log: Add tests for invalid format specifiers in strftime, and for handling of invalid file descriptors in the os module. Modified: python/trunk/Lib/test/test_datetime.py python/trunk/Lib/test/test_os.py Several of the tests added to test_os.py are invalid and fail. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
I have no idea why the connect refusal takes so long. Maybe it's a vista thing? I've looked into this further. It doesn't just happen for localhost, but also for remote hosts, and not just for IPv6, but also for IPv4, and not just for Vista, but also for XP. The problem is this: 1. Vista sends SYN to target machine (say, through v6) 2. Target machine has no port open, and responds with RST,ACK 3. Vista waits 0.5s 4. Vista sends another SYN to target machine 5. Target machine responds with another RST,ACK 6. Vista waits another 0.5s 7. Vista retries a third time, again getting no connection 8. Vista gives up, having spend 1s in trying to establish a connection to a remote port where the remote system has confirmed that the connection is refused already a second ago. I have not found documentation for this feature of the Windows TCP stack, yet. There is the TcpMaxConnectRetransmissions parameter, but this supposed affects requests without responses (and supposed also starts of with 3s) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. Kristján -Original Message- From: Martin v. Löwis [mailto:mar...@v.loewis.de] Sent: 14. janúar 2009 20:31 To: Kristján Valur Jónsson Cc: Victor Stinner; python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow I have no idea why the connect refusal takes so long. Maybe it's a vista thing? I've looked into this further. It doesn't just happen for localhost, but also for remote hosts, and not just for IPv6, but also for IPv4, and not just for Vista, but also for XP. The problem is this: 1. Vista sends SYN to target machine (say, through v6) 2. Target machine has no port open, and responds with RST,ACK 3. Vista waits 0.5s 4. Vista sends another SYN to target machine 5. Target machine responds with another RST,ACK 6. Vista waits another 0.5s 7. Vista retries a third time, again getting no connection 8. Vista gives up, having spend 1s in trying to establish a connection to a remote port where the remote system has confirmed that the connection is refused already a second ago. I have not found documentation for this feature of the Windows TCP stack, yet. There is the TcpMaxConnectRetransmissions parameter, but this supposed affects requests without responses (and supposed also starts of with 3s) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
And Microsoft, realizing their problem , came up with this: http://msdn.microsoft.com/en-us/library/bb513665(VS.85).aspx K -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Kristján Valur Jónsson Sent: 14. janúar 2009 21:40 To: Martin v. Löwis Cc: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. Kristján -Original Message- From: Martin v. Löwis [mailto:mar...@v.loewis.de] Sent: 14. janúar 2009 20:31 To: Kristján Valur Jónsson Cc: Victor Stinner; python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow I have no idea why the connect refusal takes so long. Maybe it's a vista thing? I've looked into this further. It doesn't just happen for localhost, but also for remote hosts, and not just for IPv6, but also for IPv4, and not just for Vista, but also for XP. The problem is this: 1. Vista sends SYN to target machine (say, through v6) 2. Target machine has no port open, and responds with RST,ACK 3. Vista waits 0.5s 4. Vista sends another SYN to target machine 5. Target machine responds with another RST,ACK 6. Vista waits another 0.5s 7. Vista retries a third time, again getting no connection 8. Vista gives up, having spend 1s in trying to establish a connection to a remote port where the remote system has confirmed that the connection is refused already a second ago. I have not found documentation for this feature of the Windows TCP stack, yet. There is the TcpMaxConnectRetransmissions parameter, but this supposed affects requests without responses (and supposed also starts of with 3s) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Kristján Valur Jónsson wrote: Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. I know it's pointless to express my shock here, but I can't resist. It's truly amazing to me that they'd delay the connect call's failure for a second by default, in hopes that the other end might come back up between SYN's. How often could that possibly happen? Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Support for the Haiku OS
Martin closed a patch http://bugs.python.org/issue4933 for adding support so that Python runs on Haiku. The theory is that we don't want to support minority operation systems. My view is that we should support those systems to the extent that someone like the OP is willing to maintain the handful of deltas needed to get all tests to pass (the OP's comments indicate that very few customizations are necessary). Also, I think there is some merit to supporting other open source projects that are disadvantaged only because they are small. Choices to not support them imply a choice to make their disadvantage a self-fulfilling prophecy. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Eric Smith wrote: Kristján Valur Jónsson wrote: Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. I know it's pointless to express my shock here, but I can't resist. It's truly amazing to me that they'd delay the connect call's failure for a second by default, in hopes that the other end might come back up between SYN's. How often could that possibly happen? When I read it I was tempted to observe they must have been testing Microsoft network services. It is a truly bizarre rationalization of a default that appears to have been taken from DOS-era network client applications. I remember demonstrating the phenomenon on a cli-based Telnet client at least 15 years ago. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for the Haiku OS
Raymond The theory is that we don't want to support minority operation Raymond systems. My view is that we should support those systems to Raymond the extent that someone like the OP is willing to maintain the Raymond handful of deltas needed to get all tests to pass (the OP's Raymond comments indicate that very few customizations are necessary). +1. I would argue that Haiku OS is probably no more of a minority platform at this point than OS2/EMX, which continues to be supported, at least at the level if patches being applied to the source. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Yet another undefined thing affecting us, Martin. I think it's just another case of Microsoft says it's undefined, even though the standards clearly specify what the behavior must be, and Microsoft managed to implement the behavior that not only violates the specification, but also hurts users of their systems. See Scott's analysis for why this is a case of such ignorance; I agree with this analysis, and many other people in the net also agree. Regards, Martin P.S. The sad thing is not that Microsoft makes mistakes; all system vendors do. The said thing is that they refuse to acknowledge their mistakes, and rather chose to persist the buggy implementation than risking backwards incompatibilities: somebody might rely on getting the connection only on the third attempt. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Kristján Valur Jónsson wrote: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. I think RFC793 is actually pretty clear in stating that: If the receiver [of the RST packet] was in any other state, it aborts the connection and advises the user and goes to the CLOSED state. But alas, Microsoft thinks they know better.. A brief search yields a number of threads on mailing lists for proxies having to deal with this feature. The solution that I see as most viable is temporarily making the sockets nonblocking before the connect() and scheduling our own timeout. The only variation on this is I have seen is to use GetTcpTable() to retrieve the status of the socket to determine the state of the socket (since a timeout would kill connects() that are just slow too..). -- Scott Dial sc...@scottdial.com scod...@cs.indiana.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
And Microsoft, realizing their problem , came up with this: http://msdn.microsoft.com/en-us/library/bb513665(VS.85).aspx Dual-stacked sockets are a useful thing to have (so useful that Linux made them the default, despite that the RFC says that the default should be IPV6_V6ONLY). The Python library should make all server sockets dual-stacked if possible. Unfortunately: - the socket option is not available on all systems, in particular, it is not available on Windows XP (you need Vista) - you'll see the 1s delay on the client side if the server is not dual-stacked, so if the server misbehaves, the client has to suffer. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for the Haiku OS
I'm with Martin. In these days of distributed version control systems, I would think that the effort for the Haiku folks to maintain a branch of Python in their own version control would be minimal. It is likely that for each new Python version that comes out, initially it is broken on Haiku, and then they have to go in and fix it. Doing that in their own version control has the advantage that they don't have to worry about not breaking support for any other minority operating systems, so I expect that all in all the cost will be less for them than if they have to submit these patches to core Python; and since that will definitely be less work for core Python, I would call that a win-win solution. On Wed, Jan 14, 2009 at 2:31 PM, s...@pobox.com wrote: Raymond The theory is that we don't want to support minority operation Raymond systems. My view is that we should support those systems to Raymond the extent that someone like the OP is willing to maintain the Raymond handful of deltas needed to get all tests to pass (the OP's Raymond comments indicate that very few customizations are necessary). +1. I would argue that Haiku OS is probably no more of a minority platform at this point than OS2/EMX, which continues to be supported, at least at the level if patches being applied to the source. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] stuck with dlopen...
I've recently been working on generating C functions on-the-fly which inline
the C code necessary to implement the bytecode in a given Python function.
For example, this bytecode:
dis.dis(f)
2 0 LOAD_FAST0 (a)
3 LOAD_CONST 1 (1)
6 BINARY_ADD
7 RETURN_VALUE
is transformed into this rather boring bit of C code:
#include Python.h
#include code.h
#include frameobject.h
#include eval.h
#include opcode.h
#include structmember.h
#include opcode_mini.h
PyObject *
_PyEval_EvalMiniFrameEx(PyFrameObject *f, int throwflag)
{
static int jitting = 1;
PyEval_EvalFrameEx_PROLOG1();
co = f-f_code;
PyEval_EvalFrameEx_PROLOG2();
oparg = 0;
LOAD_FAST_IMPL(oparg);
oparg = 1;
LOAD_CONST_IMPL(oparg);
BINARY_ADD_IMPL();
RETURN_VALUE_IMPL();
PyEval_EvalFrameEx_EPILOG();
}
The PROLOG1, PROLOG2 and EPILOG macros are just chunks of code from
PyEval_EvalFrameEx.
I have the code compiling and linking, and dlopen and dlsym seem to work,
returning apparently valid pointers, but when I try to call the function I
get
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x000c
0x0058066d in _PyEval_EvalMiniFrameEx (f=0x230d30, throwflag=0) at
MwDLSf.c:17
Line 17 is the PROLOG1 macro. I presume it's probably barfed on the very
first instruction. (This is all on an Intel Mac running Leopard BTW.)
Here are the commands generated to compile and link the C code:
gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall \
-Wstrict-prototypes -g -DPy_BUILD_CORE -DNDEBUG \
-I/Users/skip/src/python/py3k-t/Include \
-I/Users/skip/src/python/py3k-t -c dTd5cl.c \
-o /tmp/MwDLSf.o
gcc -L/opt/local/lib -bundle -undefined dynamic_lookup -g \
/tmp/dTd5cl.o -L/Users/skip/src/python/py3k-t -lpython3.1 \
-o /tmp/MwDLSf.so
(It just uses the distutils compiler module to build .so files.) The .so
file looks more-or-less ok:
% otool -L /tmp/MwDLSf.so
/tmp/MwDLSf.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 111.1.3)
though nm doesn't show that any undefined _Py* symbols so I suspect I'm not
linking it correctly. The Python executable was built without
--enable-shared. I've tried building with that config flag, but that just
gives me fits during debugging because it always wants to find libpython in
the installation directory even if I'm running python.exe from the build
directory. Installing is a little tedious because it relies on a properly
functioning interpreter.
dlopen is called very simply:
handle = dlopen(shared, RTLD_NOW);
I used RTLD_NOW because that's what sys.getdlopenflags() returns. I'm not
calling dlclose for the time being.
I'm not exactly sure where I should go from here. I'd be more than happy to
open an item in the issue tracker. I was hoping to get something a bit
closer to working before doing that though. The failure to properly load
the compiled function makes it pretty much impossble to debug the generated
code beyond what the compiler can tell me.
Any suggestions?
Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for the Haiku OS
On Wed, Jan 14, 2009 at 4:10 PM, Guido van Rossum gu...@python.org wrote: I'm with Martin. In these days of distributed version control systems, I would think that the effort for the Haiku folks to maintain a branch of Python in their own version control would be minimal. It is likely that for each new Python version that comes out, initially it is broken on Haiku, and then they have to go in and fix it. Doing that in their own version control has the advantage that they don't have to worry about not breaking support for any other minority operating systems, so I expect that all in all the cost will be less for them than if they have to submit these patches to core Python; and since that will definitely be less work for core Python, I would call that a win-win solution. Guido, Thanks for your feedback on this. We'd be ok with keeping track of Haiku specific patches in the HaikuPorts svn as you suggest. If we come across things we feel can apply to python as a whole and not just Haiku specific then we'll feed just those changes back to python. Ideally we'd get to the point though that each new version of python will just work on Haiku. -Scott McCreary HaikuPorts ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
