Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Martin v. Löwis
> I haven't yet tried posting a query to a FreeBSD list, as it could simply
> be a bug on amd64, but I was wondering whether there was anything (other
> than deactivating tests and documenting use of ulimit -v on this
> platform) that could be done to work around this behaviour.

I think it should be possible to debug this (for people with access to
such a system, and appropriate skills).

I find it hard to believe that a large malloc will simply crash the
process, rather than returning NULL. More likely, there is a NULL
returned somewhere, and Python (or libc) fails to check for it.

Regards,
Martni
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] RELEASED Python 2.6rc1

2008-09-17 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Barry Warsaw wrote:
> On behalf of the Python development team and the Python community, I am
> happy to announce the first release candidate for Python 2.6.

In http://www.python.org/download/releases/2.6/ , release date for
2.6rc1 is 20-Aug-2008. That is not right.

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
[EMAIL PROTECTED] - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSNDU6plgi5GaxT1NAQI0bgP8C8PcUBJA8HpSiWuE8W9Fd6p4HwzXysUQ
aA4szns+9FVKbXh0t63M0BGUM0pG4qUgy8lSQevPCxHT0DS2mXWoxAmxDWKthSUa
lU3n515KcXJ0nXr37tCsXay0YlH9qQfKU4KReG3KFQoegMzmB3kW7lwYlkk/bfsM
YmhHjM57WgA=
=sA1F
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] switching on -3 from within a program?

2008-09-17 Thread Nick Coghlan
Benjamin Peterson wrote:
> def engage_py3k_warning():
>  flag = ctypes.c_int.in_dll(ctypes.pythonapi, "Py_Py3kWarningFlag")
>  flag.value = 1

Note that tricks like this won't necessarily enable all python 3
warnings for modules that have been imported before the flag gets set.
To avoid unnecessary performance penalties as a result of Py3k warnings,
modules are permitted to do things like:

def func():
  # Do whatever

if sys.py3kwarning:
  _orig_func = func
  def func():
warnings.py3kwarn("Goes away in 3.0")
_orig_func()

Modules that are first imported after the flag has been set will
obviously have all of their warnings properly enabled.

Cheers,
Nick.


-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] switching on -3 from within a program?

2008-09-17 Thread Anthon van der Neut
Hi Nick,

I am aware of that (but others might not, so you are right to point this
out).

I did follow both Christian's and Benjamin's suggestions. The
implementation at the mod_wsgi C level, which is before any module
loading is more permanent, but the ctypes trick doesn't require an
apache2 restart, which is somewhat more comfortable.

Anthon

Nick Coghlan wrote:
> Benjamin Peterson wrote:
>> def engage_py3k_warning():
>>  flag = ctypes.c_int.in_dll(ctypes.pythonapi, "Py_Py3kWarningFlag")
>>  flag.value = 1
> 
> Note that tricks like this won't necessarily enable all python 3
> warnings for modules that have been imported before the flag gets set.
> To avoid unnecessary performance penalties as a result of Py3k warnings,
> modules are permitted to do things like:
> 
> def func():
>   # Do whatever
> 
> if sys.py3kwarning:
>   _orig_func = func
>   def func():
> warnings.py3kwarn("Goes away in 3.0")
> _orig_func()
> 
> Modules that are first imported after the flag has been set will
> obviously have all of their warnings properly enabled.
> 
> Cheers,
> Nick.
> 
> 
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] RELEASED Python 2.6rc1

2008-09-17 Thread Barry Warsaw

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sep 17, 2008, at 5:59 AM, Jesus Cea wrote:


Barry Warsaw wrote:
On behalf of the Python development team and the Python community,  
I am

happy to announce the first release candidate for Python 2.6.


In http://www.python.org/download/releases/2.6/ , release date for
2.6rc1 is 20-Aug-2008. That is not right.


Fixed, thanks.
- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSNEKK3EjvBPtnXfVAQKscgP+LXF7eWCvQxdgQ1dREvA7OeSyh60FGvBW
SHMv0aHKyZgDZlgpjOB4kttVzYNWB0mm0jHfhuYvJ60CygNVCUCjxgZx8f6SK4B/
IaMjWoFH9e4P616jwknfsvxsmZhqzRF3eyIAz3vl7U9fQWv/jsjtaQl5FcIKuH10
33xxz4TH9hU=
=S0s8
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Andrew MacIntyre

Martin v. Löwis wrote:

I haven't yet tried posting a query to a FreeBSD list, as it could simply
be a bug on amd64, but I was wondering whether there was anything (other
than deactivating tests and documenting use of ulimit -v on this
platform) that could be done to work around this behaviour.


I think it should be possible to debug this (for people with access to
such a system, and appropriate skills).

I find it hard to believe that a large malloc will simply crash the
process, rather than returning NULL. More likely, there is a NULL
returned somewhere, and Python (or libc) fails to check for it.


A simple C program doing a repetitive malloc(), much as pymalloc would
with continuous demand, does indeed not see any NULL from malloc() when
swap is exhausted but the process gets KILLed (the allocated memory does
have to be written to to force the situation...)

I'll take this up with FreeBSD folk, but I'm open to ideas as to how
best to deal with the problem in the context of the test suite pending
resolution by FreeBSD.

Regards,
Andrew.

--
-
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: [EMAIL PROTECTED]  (pref) | Snail: PO Box 370
   [EMAIL PROTECTED] (alt) |Belconnen ACT 2616
Web:http://www.andymac.org/   |Australia

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread James Y Knight


On Sep 17, 2008, at 10:53 AM, Andrew MacIntyre wrote:


Martin v. Löwis wrote:
I haven't yet tried posting a query to a FreeBSD list, as it could  
simply
be a bug on amd64, but I was wondering whether there was anything  
(other

than deactivating tests and documenting use of ulimit -v on this
platform) that could be done to work around this behaviour.
I think it should be possible to debug this (for people with access  
to

such a system, and appropriate skills).
I find it hard to believe that a large malloc will simply crash the
process, rather than returning NULL. More likely, there is a NULL
returned somewhere, and Python (or libc) fails to check for it.


A simple C program doing a repetitive malloc(), much as pymalloc would
with continuous demand, does indeed not see any NULL from malloc()  
when
swap is exhausted but the process gets KILLed (the allocated memory  
does

have to be written to to force the situation...)

I'll take this up with FreeBSD folk, but I'm open to ideas as to how
best to deal with the problem in the context of the test suite pending
resolution by FreeBSD.


Linux does the same thing, unless the user has explicitly configured  
that behavior off. Search the web for linux overcommit. It's  
controlled by the vm.overcommit_memory sysctl. Although linux's  
default is some heuristic that might make Python's test case work  
right in most cases, depending on malloc returning NULL is not  
something you can actually depend on.


James
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Alex Martelli
Unbelievable as this may seem, this crazy over-committing malloc
behavior is by now "a classic" -- I first fought against it in 1990,
when IBM released AIX 3 for its then-new RS/6000 line of workstations;
in a later minor release they did provide a way to optionally switch
this off, but, like on Linux, it's a system-wide switch, NOT
per-process:-(.

I concur with http://www.win.tue.nl/~aeb/linux/lk/lk-9.html (the best
explanation I know of the subject, and recommended reading) which, on
this subject, says "Linux on the other hand is seriously broken."
(just like AIX 3 was).  Sad to learn that BSD is now also broken in
the same way:-(.


Alex
On Wed, Sep 17, 2008 at 8:00 AM, James Y Knight <[EMAIL PROTECTED]> wrote:
>
> On Sep 17, 2008, at 10:53 AM, Andrew MacIntyre wrote:
>
>> Martin v. Löwis wrote:

 I haven't yet tried posting a query to a FreeBSD list, as it could
 simply
 be a bug on amd64, but I was wondering whether there was anything (other
 than deactivating tests and documenting use of ulimit -v on this
 platform) that could be done to work around this behaviour.
>>>
>>> I think it should be possible to debug this (for people with access to
>>> such a system, and appropriate skills).
>>> I find it hard to believe that a large malloc will simply crash the
>>> process, rather than returning NULL. More likely, there is a NULL
>>> returned somewhere, and Python (or libc) fails to check for it.
>>
>> A simple C program doing a repetitive malloc(), much as pymalloc would
>> with continuous demand, does indeed not see any NULL from malloc() when
>> swap is exhausted but the process gets KILLed (the allocated memory does
>> have to be written to to force the situation...)
>>
>> I'll take this up with FreeBSD folk, but I'm open to ideas as to how
>> best to deal with the problem in the context of the test suite pending
>> resolution by FreeBSD.
>
> Linux does the same thing, unless the user has explicitly configured that
> behavior off. Search the web for linux overcommit. It's controlled by the
> vm.overcommit_memory sysctl. Although linux's default is some heuristic that
> might make Python's test case work right in most cases, depending on malloc
> returning NULL is not something you can actually depend on.
>
> James
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Martin v. Löwis
> I'll take this up with FreeBSD folk, but I'm open to ideas as to how
> best to deal with the problem in the context of the test suite pending
> resolution by FreeBSD.

Not sure what the test purpose is: if it is to test that you get a
MemoryError in cases where you ask for more than Python could represent,
and the test errs on system where the requested size is actually
representable, the solution then is to fix the test case.

If the test purpose is to trigger a memory error for cases when the
system runs out of memory, the test case should set a ulimit to less
than the physical memory.

It might be useful to have an interpreter-maintained limit on the amount
of memory Python can consume, but such a feature is clearly out of scope
for the current state of the code.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Jon Ribbens
On Wed, Sep 17, 2008 at 08:21:55AM -0700, Alex Martelli wrote:
> Unbelievable as this may seem, this crazy over-committing malloc
> behavior is by now "a classic" -- I first fought against it in 1990,
> when IBM released AIX 3 for its then-new RS/6000 line of workstations;
> in a later minor release they did provide a way to optionally switch
> this off, but, like on Linux, it's a system-wide switch, NOT
> per-process:-(.
> 
> I concur with http://www.win.tue.nl/~aeb/linux/lk/lk-9.html (the best
> explanation I know of the subject, and recommended reading) which, on
> this subject, says "Linux on the other hand is seriously broken."
> (just like AIX 3 was).  Sad to learn that BSD is now also broken in
> the same way:-(.

It's now "now" also broken, it has been that way for a very long time.
For example, see this message I wrote back in July 1999 complaining
about FreeBSD overcommit:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg01056.html
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Bill Janssen
Giampaolo Rodola' <[EMAIL PROTECTED]> wrote:

> 2 - By reading ssl.py code I noticed that when do_handshake_on_connect
> flag is False the do_handshake() method is never called. Is it
> supposed to be manually called when dealing with non-blocking sockets?

Yes.  Look at the example client in Lib/test/test_ssl.py.  The server
code should work the same way.

Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Bill Janssen
Giampaolo Rodola' <[EMAIL PROTECTED]> wrote:

> I change my question: how am I supposed to know when the SSL hanshake
> is completed? When pending() returns False?

When do_handshake() doesn't raise an exception.

Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Bill Janssen
Giampaolo Rodola' <[EMAIL PROTECTED]> wrote:

> In the meanwhile I noticed something in the ssl.py code which seems to
> be wrong:
> 
> def recv (self, buflen=1024, flags=0):
> if self._sslobj:
> if flags != 0:
> raise ValueError(
> "non-zero flags not allowed in calls to sendall()
> on %s" %
> self.__class__)
> while True:
> try:
> return self.read(buflen)
> except SSLError, x:
> if x.args[0] == SSL_ERROR_WANT_READ:
> continue
> else:
> raise x
> else:
> return socket.recv(self, buflen, flags)
> 
> I don't know the low levels but that while statement which continues
> in case of SSL_ERROR_WANT_READ seems to be wrong (blocking), at least
> when dealing with non-blocking sockets. I think the proper way of
> doing recv() here is letting SSL_ERROR_WANT_READ propagate and let the
> upper application (e.g. asyncore) deal with it.

It's an interesting point.  I'm not sure the underlying code will ever
raise this exception.  Please file a bug report to help us track this.

Thanks.

Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] RELEASED Python 2.6rc1

2008-09-17 Thread Arfrever Frehtes Taifersar Arahesis
2008-09-17 15:46:19 Barry Warsaw napisał(a):
> On Sep 17, 2008, at 5:59 AM, Jesus Cea wrote:
> >
> > Barry Warsaw wrote:
> >> On behalf of the Python development team and the Python community,
> >> I am
> >> happy to announce the first release candidate for Python 2.6.
> >
> > In http://www.python.org/download/releases/2.6/ , release date for
> > 2.6rc1 is 20-Aug-2008. That is not right.
> 
> Fixed, thanks.

Release date for 2.6rc1 should be 12-Sep-2008 instead of 17-Sep-2008.

-- 
Arfrever Frehtes Taifersar Arahesis


signature.asc
Description: This is a digitally signed message part.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Bill Janssen
Ah, now I remember.  It seems that sometimes when SSL_ERROR_WANT_READ
was returned, things would block; that is, the "handle_read" method on
asyncore.dispatcher was never called again, so the SSLSocket.recv()
method was never re-called.  There are several levels of buffering going
on, and I never figured out just why that was.  This (very rare) re-call
of "read" is to handle that.

Bill

Bill Janssen <[EMAIL PROTECTED]> wrote:

> Giampaolo Rodola' <[EMAIL PROTECTED]> wrote:
> 
> > In the meanwhile I noticed something in the ssl.py code which seems to
> > be wrong:
> > 
> > def recv (self, buflen=1024, flags=0):
> > if self._sslobj:
> > if flags != 0:
> > raise ValueError(
> > "non-zero flags not allowed in calls to sendall()
> > on %s" %
> > self.__class__)
> > while True:
> > try:
> > return self.read(buflen)
> > except SSLError, x:
> > if x.args[0] == SSL_ERROR_WANT_READ:
> > continue
> > else:
> > raise x
> > else:
> > return socket.recv(self, buflen, flags)
> > 
> > I don't know the low levels but that while statement which continues
> > in case of SSL_ERROR_WANT_READ seems to be wrong (blocking), at least
> > when dealing with non-blocking sockets. I think the proper way of
> > doing recv() here is letting SSL_ERROR_WANT_READ propagate and let the
> > upper application (e.g. asyncore) deal with it.
> 
> It's an interesting point.  I'm not sure the underlying code will ever
> raise this exception.  Please file a bug report to help us track this.
> 
> Thanks.
> 
> Bill
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/janssen%40parc.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Jean-Paul Calderone

On Wed, 17 Sep 2008 10:40:01 PDT, Bill Janssen <[EMAIL PROTECTED]> wrote:

Ah, now I remember.  It seems that sometimes when SSL_ERROR_WANT_READ
was returned, things would block; that is, the "handle_read" method on
asyncore.dispatcher was never called again, so the SSLSocket.recv()
method was never re-called.  There are several levels of buffering going
on, and I never figured out just why that was.  This (very rare) re-call
of "read" is to handle that.



You certainly do need to call read again if OpenSSL fails an SSL_read with
a want-read error, but in asyncore, you don't want to do it right away,
you want to wait until the socket becomes readable again, otherwise you *do*
block waiting for bytes from the network.  See the SSL support in Twisted
for an example of the correct way to handle this.

Jean-Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Bill Janssen
Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:

> On Wed, 17 Sep 2008 10:40:01 PDT, Bill Janssen <[EMAIL PROTECTED]> wrote:
> >Ah, now I remember.  It seems that sometimes when SSL_ERROR_WANT_READ
> >was returned, things would block; that is, the "handle_read" method on
> >asyncore.dispatcher was never called again, so the SSLSocket.recv()
> >method was never re-called.  There are several levels of buffering going
> >on, and I never figured out just why that was.  This (very rare) re-call
> >of "read" is to handle that.
> >
> 
> You certainly do need to call read again if OpenSSL fails an SSL_read with
> a want-read error, but in asyncore, you don't want to do it right away,
> you want to wait until the socket becomes readable again, otherwise you *do*
> block waiting for bytes from the network.  See the SSL support in Twisted
> for an example of the correct way to handle this.
> 
> Jean-Paul

Yes, I understand, and that's how I started out.  The bug we were seeing
was that "handle_read" wasn't being called again by asyncore.

Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Leonardo Santagada


On Sep 17, 2008, at 12:45 PM, Martin v. Löwis wrote:


I'll take this up with FreeBSD folk, but I'm open to ideas as to how
best to deal with the problem in the context of the test suite  
pending

resolution by FreeBSD.


Not sure what the test purpose is: if it is to test that you get a
MemoryError in cases where you ask for more than Python could  
represent,

and the test errs on system where the requested size is actually
representable, the solution then is to fix the test case.

If the test purpose is to trigger a memory error for cases when the
system runs out of memory, the test case should set a ulimit to less
than the physical memory.

It might be useful to have an interpreter-maintained limit on the  
amount
of memory Python can consume, but such a feature is clearly out of  
scope

for the current state of the code.



There is an option at least on linux to hack using ld preload to use  
another memory manager that respond the way needed... at least that  
was what I was told today at lunch. (if ulimit is not enough for any  
reason).



--
Leonardo Santagada
santagada at gmail.com



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] FreeBSD 7 amd64 and large memory tests

2008-09-17 Thread Martin v. Löwis
> There is an option at least on linux to hack using ld preload to use
> another memory manager that respond the way needed... at least that was
> what I was told today at lunch. (if ulimit is not enough for any reason).

For Python, there would be much less hackish ways. Most if not all calls
to memory allocators are channeled through Python API, which could be
diverted on source level also.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Nick Coghlan
Bill Janssen wrote:
> Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> 
>> On Wed, 17 Sep 2008 10:40:01 PDT, Bill Janssen <[EMAIL PROTECTED]> wrote:
>>> Ah, now I remember.  It seems that sometimes when SSL_ERROR_WANT_READ
>>> was returned, things would block; that is, the "handle_read" method on
>>> asyncore.dispatcher was never called again, so the SSLSocket.recv()
>>> method was never re-called.  There are several levels of buffering going
>>> on, and I never figured out just why that was.  This (very rare) re-call
>>> of "read" is to handle that.
>>>
>> You certainly do need to call read again if OpenSSL fails an SSL_read with
>> a want-read error, but in asyncore, you don't want to do it right away,
>> you want to wait until the socket becomes readable again, otherwise you *do*
>> block waiting for bytes from the network.  See the SSL support in Twisted
>> for an example of the correct way to handle this.
>>
>> Jean-Paul
> 
> Yes, I understand, and that's how I started out.  The bug we were seeing
> was that "handle_read" wasn't being called again by asyncore.

It's probably worth sticking a comment in the code explaining why we're
taking the risk of temporarily blocking on a non-blocking socket (i.e.
until someone figures out how to reproduce that bug reliably so that a
more correct fix can be devised).

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ssl module, non-blocking sockets and asyncore integration

2008-09-17 Thread Giampaolo Rodola'
Ok, here's some news, in case they can be of some interest.
I managed to write an asyncore disptacher which seems to work.
I used my test suite against it and 70 tests passed and 2 failed.
The tests failed because at a certain point a call to do_handhsake
results in an EOF exception, which is very strange since it is
supposed to raise SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE only.
I'll keep you updated in case I have some news.



--- Exception ---

  File "C:\python26\lib\ssl.py", line 293, in do_handshake
self._sslobj.do_handshake()
SSLError: [Errno 8] _ssl.c:480: EOF occurred in violation of protocol



--- SSL dispatcher 

class SSLConnection(asyncore.dispatcher):

def __init__(self):
self.ssl_handshake_pending = False

def do_ssl_handshake(self):
try:
self.socket.do_handshake()
except ssl.SSLError, err:
if err.args[0] == ssl.SSL_ERROR_WANT_READ:
self.ssl_handshake_pending = 'read'
elif err.args[0] == ssl.SSL_ERROR_WANT_WRITE:
self.ssl_handshake_pending = 'write'
else:
raise
else:
self.ssl_handshake_pending = False

def handle_read_event(self):
if self.ssl_handshake_pending == 'read':
self.do_ssl_handshake()
##if not self.ssl_handshake_pending:
##asyncore.dispatcher.handle_read_event(self)
else:
asyncore.dispatcher.handle_read_event(self)

def handle_write_event(self):
if self.ssl_handshake_pending == 'write':
self.do_ssl_handshake()
##if not self.ssl_handshake_pending:
##asyncore.dispatcher.handle_write_event(self)
else:
asyncore.dispatcher.handle_write_event(self)



--- Giampaolo
http://code.google.com/p/pyftpdlib/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] switching on -3 from within a program?

2008-09-17 Thread Graham Dumpleton
On Sep 17, 1:32 am, Anthon van der Neut <[EMAIL PROTECTED]> wrote:
> With a minor modification to the Makefile I was able to getmod_wsgi
> v2.3 to work with python2.6rc1.

Which I believe was still required only because you have only
installed static Python library. :-)

> I promptly got a warning in my apache log files on the depcrecated use
> of 'sha' in the paste's cookie module, good! And easily fixed.
>
> After that I was looking for a way to switch on the -3 warnings from
> within my code to have extra warnings show up in my apache log file.

Check out mod_wsgi from subversion repository:

 svn co http://modwsgi.googlecode.com/svn/trunk/mod_wsgi

and build that.

When Pyhton 2.6 is being used, you should be able to have in Apache
configuration file at server scope.

  WSGIPy3kWarningFlag On

You must not be loading mod_python at same time else this will not be
honoured.

Graham

> After reading some documentation I tried from the python2.6 prompt:
>
> import sys
> sys.py3kwarning = True
> x = { 'abc': 1 }; x.has_key('abc')
>
> which does not give a warning (starting python2.6 with the -3 option of
> course does).
>
> Is there anyway to switch this on from within a program with a Python
> statement?
> If not, would I need to make a small C-extension module (to be imported
> as the first module) that sets Py_Py3WarningFlag or something else at
> the C level, or would that better be done bymod_wsgi'sC code.
>
> Regards
> Anthon
> ___
> Python-Dev mailing list
> [EMAIL PROTECTED]://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] RELEASED Python 2.6rc2 and 3.0rc1

2008-09-17 Thread Barry Warsaw

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On behalf of the Python development team and the Python community, I  
am happy to announce the second and final planned release candidate  
for Python 2.6, as well as the first release candidate for Python 3.0.


These are release candidates, so while they are not suitable for  
production environments, we strongly encourage you to download and  
test them on your software.  We expect only critical bugs to be fixed  
between now and the final releases.  Currently Python 2.6 is scheduled  
for October 1st, 2008.  Python 3.0 release candidate 2 is planned for  
October 1st, with the final release planned for October 15, 2008.


If you find things broken or incorrect, please submit bug reports at

http://bugs.python.org

For more information and downloadable distributions, see the Python  
2.6 website:


http://www.python.org/download/releases/2.6/

and the Python 3.0 web site:

http://www.python.org/download/releases/3.0/

See PEP 361 for release schedule details:

http://www.python.org/dev/peps/pep-0361/

Enjoy,
- -Barry

Barry Warsaw
[EMAIL PROTECTED]
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSNHpw3EjvBPtnXfVAQLW9wP/RBCaUvhuheIh+BjLLIHQFBQi7D3uVgqi
l0+4fhhoKGJvtWklLfSM9I1prcjH/d6tzUu4fIOjX7aM+wZSG++vkfmBoehnhyZW
AvU9Lax4mqDwhOJA2QA0WMx0obpYYVHeUl7D1g9kWzbRUkZDX9NZGMWThhEOC1qA
UA3bBYbvWiQ=
=BFNH
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com