Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Tres Seaver wrote:
> Guido van Rossum wrote:
>> On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen  wrote:
>>> Josiah Carlson  wrote:
>>>
 But yes, zope needs to be changed to reflect the updated
 asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
 easier to use now.
>>> Just for completeness, I built a fresh 2.6, installed Medusa (from
>>> http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
>>> as well as it does on 2.5, anyway).  I think this is a Zope issue.
>> Way back in the day, Zope monkeypatched whole parts of asyncore,
>> replacing them with some of its own code. If that's still the case,
>> this breakage should be no surprise.
> 
> Zope has been hooking the 'asyncore.loop' function (wrapping it in order
> to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is
> the earliest checkout I have, and it was branched in early January 2002).
> 
> Zope also began patched asyncore's logging functions in 2.7, and later
> updated that patch to make the logger use the stdlib 'logging' module.
> 
> I think the change we need to make (in ZServer/medusa/http_server.py) is
> to emulate the new 'writeable' implementation in asynchat, at least when
> running under 2.6.
> 
> Zope's 'ZServer.medusa.http_server.http_request.writable()'
> implementation is::
> 
> def writable (self):
> # this is just the normal async_chat 'writable',
> # here for comparison
> return self.ac_out_buffer or len(self.producer_fifo)
> 
> 
> The Python 2.5 asynchat.asynchat.writable does::
> 
> def writable (self):
> "predicate for inclusion in the writable for select()"
> # return len(self.ac_out_buffer) or len(self.producer_fifo) or
> # (not self.connected)
> # this is about twice as fast, though not as clear.
> return not (
> (self.ac_out_buffer == '') and
> self.producer_fifo.is_empty() and
> self.connected
> )
> 
> The Python 2.6 asynchat.asynchat.writable now does::
> 
> def writable (self):
> "predicate for inclusion in the writable for select()"
> return self.producer_fifo or (not self.connected)
> 
> 
> medusa 0.5.4's medusa.http_server.http_request doesn't override
> 'writable', but it does have a broken 'writeable_for_proxy':
> 
> 
> def writable_for_proxy (self):
> # this version of writable supports the idea of a 'stalled'
> # producer
> # [i.e., it's not ready to produce any output yet] This is
> # needed by
> # the proxy, which will be waiting for the magic combination of
> # 1) hostname resolved
> # 2) connection made
> # 3) data available.
> if self.ac_out_buffer:
> return 1
> elif len(self.producer_fifo):
> p = self.producer_fifo.first()
> if hasattr (p, 'stalled'):
> return not p.stalled()
> else:
> return 1

*crickets*.

The 2.6 version of asyncore *also* breaks supervisord, which does *not*
use Zope's medusa:  it uses medusa 0.5.4.  How is anybody supposed to
write a package which sits atop a library like asyncore in a fashion
portable across Python versions?  The changes to the implementation in
2.6 (there is no real API) can't be reconciled, AFAICT.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJrZG0+gerLs4ltQ4RAnsmAJ9v/vPkHgE3AdP5ngVuYaKlxDGhJACgsCi2
3awbUffi2BU41qQgd6eJV18=
=WBt6
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver  wrote:
> How is anybody supposed to
> write a package which sits atop a library like asyncore in a fashion
> portable across Python versions?  The changes to the implementation in
> 2.6 (there is no real API) can't be reconciled, AFAICT.

This seems to be the crux of the problem with asyncore, ever since it
was added to the stdlib -- there's no real API, so every change
potentially breaks something. I wish we could start over with a proper
design under a new name. Maybe packages requiring asyncore
functionality should just copy the version of asyncore they like into
their own three and stick with that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 08:46 pm, gu...@python.org wrote:

This seems to be the crux of the problem with asyncore, ever since it
was added to the stdlib -- there's no real API, so every change
potentially breaks something. I wish we could start over with a proper
design under a new name.


Might I suggest "reactor"... or possibly "twisted", as that new name? 
;-)


(Sorry, I was trying to avoid this thread, but that was an opening I 
could drive a truck through).


In all seriousness, I seem to recall that Thomas Wouters was interested 
in doing integrating some portion of Twisted core into the standard 
library as of last PyCon.  I mention him specifically by name in the 
hopes that it will jog his memory.


At the very least, this might serve as a basis for an abstract API for 
asyncore:


http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Guido van Rossum wrote:
> On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver  wrote:
>> How is anybody supposed to
>> write a package which sits atop a library like asyncore in a fashion
>> portable across Python versions?  The changes to the implementation in
>> 2.6 (there is no real API) can't be reconciled, AFAICT.
> 
> This seems to be the crux of the problem with asyncore, ever since it
> was added to the stdlib -- there's no real API, so every change
> potentially breaks something. I wish we could start over with a proper
> design under a new name. Maybe packages requiring asyncore
> functionality should just copy the version of asyncore they like into
> their own three and stick with that.

That was the very solution Chris came up with earlier today:  "stick a
fork in it, its done!"


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJraty+gerLs4ltQ4RAjBSAJ4niecZJusKY4XiioJ18mdhdMixxQCfWvcQ
Dwkh1ZBuxtGRbhUI4qy96Sc=
=ms0I
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Tue, Mar 3, 2009 at 1:17 PM,   wrote:
>
> On 08:46 pm, gu...@python.org wrote:
>>
>> This seems to be the crux of the problem with asyncore, ever since it
>> was added to the stdlib -- there's no real API, so every change
>> potentially breaks something. I wish we could start over with a proper
>> design under a new name.
>
> Might I suggest "reactor"... or possibly "twisted", as that new name? ;-)
>
> (Sorry, I was trying to avoid this thread, but that was an opening I could
> drive a truck through).
>
> In all seriousness, I seem to recall that Thomas Wouters was interested in
> doing integrating some portion of Twisted core into the standard library as
> of last PyCon.  I mention him specifically by name in the hopes that it will
> jog his memory.
>
> At the very least, this might serve as a basis for an abstract API for
> asyncore:
>
> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html

I hope we have learned from asyncore that a future-proof and maximally
flexible design of such an API is not easy. This is one of those cases
where I believe thinking hard up front will really pay off. In this
case I would recommend going through a full-blown PEP process before
committing to an API, to avoid making the same mistake twice. (I don't
mean this as a dig at Twisted, just as an observation about this
particular design problem. If Twisted solves all the problems already,
writing the PEP should be easy.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 3 Mar, 10:20 pm, gu...@python.org wrote:

On Tue, Mar 3, 2009 at 1:17 PM,   wrote:



At the very least, this might serve as a basis for an abstract API for
asyncore:

http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html



I hope we have learned from asyncore that a future-proof and maximally
flexible design of such an API is not easy. This is one of those cases
where I believe thinking hard up front will really pay off. In this
case I would recommend going through a full-blown PEP process before
committing to an API, to avoid making the same mistake twice. (I don't
mean this as a dig at Twisted, just as an observation about this
particular design problem. If Twisted solves all the problems already,
writing the PEP should be easy.)


I didn't see it as a dig at Twisted; you're right, this is a hard 
problem.  Twisted doesn't solve all the problems perfectly, either; much 
has been made of some rather unfortunate nuances of the IProducer and 
IConsumer interfaces.


I also regret that no Twisted people (myself included) seem to have the 
time to go through the PEP process and get something accepted.


Even if Twisted's interfaces aren't perfect, there's a lot of working 
application code which demonstrates that they are sufficient.  Perhaps 
more importantly, they are future-proof.  Every proposal we've come up 
with for addressing the more subtle shortcomings of these APIs has been 
completely backwards-compatible; there are no APIs with vague or broken 
semantics, just sub-optimal spellings.  The fact that there are already 
multiple reactor implementations which share very little code helps.


If someone who has PEP-writing skills and some free time is updating 
asyncore to be more modern, I'd be very happy to walk that person 
through Twisted's API design and explain how either portions of our 
implementation could be lifted for the stdlib, or how a "high level" 
layer could be written into asyncore so that the author of a particular 
chunk of low-level networking code could ignore whether they're using 
the stdlib mainloop or one of the Twisted ones.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Tue, Mar 3, 2009 at 9:31 PM,   wrote:
>
> On 3 Mar, 10:20 pm, gu...@python.org wrote:
>>
>> On Tue, Mar 3, 2009 at 1:17 PM,   wrote:
>
>>> At the very least, this might serve as a basis for an abstract API for
>>> asyncore:
>>>
>>>
>>> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html
>
>> I hope we have learned from asyncore that a future-proof and maximally
>> flexible design of such an API is not easy. This is one of those cases
>> where I believe thinking hard up front will really pay off. In this
>> case I would recommend going through a full-blown PEP process before
>> committing to an API, to avoid making the same mistake twice. (I don't
>> mean this as a dig at Twisted, just as an observation about this
>> particular design problem. If Twisted solves all the problems already,
>> writing the PEP should be easy.)
>
> I didn't see it as a dig at Twisted; you're right, this is a hard problem.
>  Twisted doesn't solve all the problems perfectly, either; much has been
> made of some rather unfortunate nuances of the IProducer and IConsumer
> interfaces.
>
> I also regret that no Twisted people (myself included) seem to have the time
> to go through the PEP process and get something accepted.

Same here.

> Even if Twisted's interfaces aren't perfect, there's a lot of working
> application code which demonstrates that they are sufficient.  Perhaps more
> importantly, they are future-proof.  Every proposal we've come up with for
> addressing the more subtle shortcomings of these APIs has been completely
> backwards-compatible; there are no APIs with vague or broken semantics, just
> sub-optimal spellings.  The fact that there are already multiple reactor
> implementations which share very little code helps.

Sounds like it's not so much the code that's future proof but the
process used for evolving it. That seems to be missing for asyncore.
:-(

> If someone who has PEP-writing skills and some free time is updating
> asyncore to be more modern, I'd be very happy to walk that person through
> Twisted's API design and explain how either portions of our implementation
> could be lifted for the stdlib, or how a "high level" layer could be written
> into asyncore so that the author of a particular chunk of low-level
> networking code could ignore whether they're using the stdlib mainloop or
> one of the Twisted ones.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Sidnei da Silva]

On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum  wrote:
> Sounds like it's not so much the code that's future proof but the
> process used for evolving it. That seems to be missing for asyncore.
> :-(

Turning the issue around a bit, has anyone considered polishing up the
current fix to restore it's backwards compatibility, instead of
starting a discussion about a full-blown replacement?

I think that would be enough for most asyncore users (or even the
couple few affected) for the moment, and then we can think about a
possible future replacement.

-- 
Sidnei da Silva
Canonical Ltd.
 Landscape · Changing the way you manage your systems
http://landscape.canonical.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
 wrote:
> On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum  wrote:
>> Sounds like it's not so much the code that's future proof but the
>> process used for evolving it. That seems to be missing for asyncore.
>> :-(
>
> Turning the issue around a bit, has anyone considered polishing up the
> current fix to restore it's backwards compatibility, instead of
> starting a discussion about a full-blown replacement?
>
> I think that would be enough for most asyncore users (or even the
> couple few affected) for the moment, and then we can think about a
> possible future replacement.

If it can be done while maintaining backwards compatibility with both
the 2.6 version and the pre-2.6 version, that would be great of
course! But can it?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Jean-Paul Calderone]

On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum  wrote:

On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
 wrote:

On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum  wrote:

Sounds like it's not so much the code that's future proof but the
process used for evolving it. That seems to be missing for asyncore.
:-(


Turning the issue around a bit, has anyone considered polishing up the
current fix to restore it's backwards compatibility, instead of
starting a discussion about a full-blown replacement?

I think that would be enough for most asyncore users (or even the
couple few affected) for the moment, and then we can think about a
possible future replacement.


If it can be done while maintaining backwards compatibility with both
the 2.6 version and the pre-2.6 version, that would be great of
course! But can it?



Is it really necessary to retain compatibility with the Python 2.6
version?  Python 2.6.0 and Python 2.6.1 contain a regression (as
compared to basically all previous versions of Python) which prevents
asyncore-based programs which are years old from working on them.
Restoring the pre-2.6 behavior will fix these old, presumably stable,
widely used programs for users who install 2.6.2 and newer.

The downside (which you were imagining, I'm sure) is that any new
software developed against the Python 2.6.0 or 2.6.1 behavior will
then break in 2.6.2 and later.  While this is unfortunate, it is
clearly the far lesser of two evils.  The choice must be made, though.
Either leave old software broken or break new software.  Just because
the "leave old software broken" choice is made through inaction doesn't
make it the better choice (though obviously since it requires action,
someone will have to do it, and I'm not volunteering - if inaction is
the choice because no one wants to do the work, fine, but that's a
different motivation than avoiding breaking newly written software).

So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone  wrote:
> - Show quoted text -
> On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum 
> wrote:
>>
>> On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
>>  wrote:
>>>
>>> On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum 
>>> wrote:

 Sounds like it's not so much the code that's future proof but the
 process used for evolving it. That seems to be missing for asyncore.
 :-(
>>>
>>> Turning the issue around a bit, has anyone considered polishing up the
>>> current fix to restore it's backwards compatibility, instead of
>>> starting a discussion about a full-blown replacement?
>>>
>>> I think that would be enough for most asyncore users (or even the
>>> couple few affected) for the moment, and then we can think about a
>>> possible future replacement.
>>
>> If it can be done while maintaining backwards compatibility with both
>> the 2.6 version and the pre-2.6 version, that would be great of
>> course! But can it?
>>
>
> Is it really necessary to retain compatibility with the Python 2.6
> version?  Python 2.6.0 and Python 2.6.1 contain a regression (as
> compared to basically all previous versions of Python) which prevents
> asyncore-based programs which are years old from working on them.
> Restoring the pre-2.6 behavior will fix these old, presumably stable,
> widely used programs for users who install 2.6.2 and newer.
>
> The downside (which you were imagining, I'm sure) is that any new
> software developed against the Python 2.6.0 or 2.6.1 behavior will
> then break in 2.6.2 and later.  While this is unfortunate, it is
> clearly the far lesser of two evils.  The choice must be made, though.
> Either leave old software broken or break new software.  Just because
> the "leave old software broken" choice is made through inaction doesn't
> make it the better choice (though obviously since it requires action,
> someone will have to do it, and I'm not volunteering - if inaction is
> the choice because no one wants to do the work, fine, but that's a
> different motivation than avoiding breaking newly written software).
>
> So, as a disinterested party in this specific case, I'd say revert to
> the pre-2.6 behavior.  It does less harm than leaving the current
> behavior.

Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this "API" is designed to work in 2.5 as well. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Jean-Paul Calderone]

On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum  wrote:

On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone  wrote:
[snip]


So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.


Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this "API" is designed to work in 2.5 as well. :-)



But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility within
the 2.6 series in this case (sorry if it appeared earlier in this thread
and I missed it)?

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone  wrote:
> On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum 
> wrote:
>>
>> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
>> wrote:
>> [snip]
>>>
>>> So, as a disinterested party in this specific case, I'd say revert to
>>> the pre-2.6 behavior.  It does less harm than leaving the current
>>> behavior.
>>
>> Sorry, but I really do think that we should maintain backward
>> compatibility *within* the 2.6 series as well. If that makes it
>> impossible to also maintain the 2.5 behavior, perhaps some flag could
>> be added to restore 2.5 compatibility, e.g.
>>
>> import asyncore
>> asyncore.python_25_compat = True
>>
>> Note that this "API" is designed to work in 2.5 as well. :-)

> But why?  The argument I made had the objective of minimizing developer
> effort.  What's the objective of maintaining backward compatibility within
> the 2.6 series in this case (sorry if it appeared earlier in this thread
> and I missed it)?

The same as always. We don't change APIs in bugfix releases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Jean-Paul Calderone]

On Wed, 4 Mar 2009 10:54:19 -0800, Guido van Rossum  wrote:

On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone  wrote:

On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum 
wrote:


On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
wrote:
[snip]


So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.


Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this "API" is designed to work in 2.5 as well. :-)



But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility within
the 2.6 series in this case (sorry if it appeared earlier in this thread
and I missed it)?


The same as always. We don't change APIs in bugfix releases.



Okay.  Thanks for explaining.

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Steve Holden]

Guido van Rossum wrote:
> On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone  
> wrote:
>> On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum 
>> wrote:
>>> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
>>> wrote:
>>> [snip]
 So, as a disinterested party in this specific case, I'd say revert to
 the pre-2.6 behavior.  It does less harm than leaving the current
 behavior.
>>> Sorry, but I really do think that we should maintain backward
>>> compatibility *within* the 2.6 series as well. If that makes it
>>> impossible to also maintain the 2.5 behavior, perhaps some flag could
>>> be added to restore 2.5 compatibility, e.g.
>>>
>>> import asyncore
>>> asyncore.python_25_compat = True
>>>
>>> Note that this "API" is designed to work in 2.5 as well. :-)
> 
>> But why?  The argument I made had the objective of minimizing developer
>> effort.  What's the objective of maintaining backward compatibility within
>> the 2.6 series in this case (sorry if it appeared earlier in this thread
>> and I missed it)?
> 
> The same as always. We don't change APIs in bugfix releases.
> 
Which unfortunately stops us from categorizing API breakages as bugs.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 06:51 pm, exar...@divmod.com wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum  
wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
 wrote:



So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.



Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well.



But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility 
within
the 2.6 series in this case (sorry if it appeared earlier in this 
thread

and I missed it)?


Python's compatibility policy dictates that compatibility between micro- 
version releases be maintained *extremely* carefully.


As you all well know, normally I'd be strongly in favor of adhering to 
such a policy.  However, the ostensible purpose of the policy here is 
that users can *always* install micro-version upgrades to get bugfixes 
and security fixes, without worrying about compatibility upgrades.


Maintaining compatibility with the 2.6.x version of asyncore presupposes 
that *someone* has written some software against that version of 
asyncore and it might break if they installed an upgrade, though.  If 
that's the case - if there's even one person who has written or run any 
asyncore software against the version in 2.6 - then I think maintaining 
bug-for-bug compatibility is important.


However, my guess - and I'm assuming that JP was thinking the same thing 
- is that literally nobody has done that, or even *would* ever do that, 
so there's no software anywhere in the world that would break.  asyncore 
is relatively unpopular (thanks in part to the excellent alternatives 
:-)); its major users are the same people who have already complained.


Maybe it's too late to do something like this for 2.6, or even 3.0, but 
I've thought on a few occasions that projects (both Python and Twisted, 
at least) could use a special very-low-traffic policy-deviations mailing 
list for "We *REALLY* don't think this is going to ever break anything 
anyone has actually done, but just to be sure..." notifications for 
situations like this.  The implications being that (A) _everyone_ who 
uses the software should subscribe, and (B) if anyone ever responded to 
a message, the deviation wouldn't take place.


Anyway, I'm also a disinterested observer, so please don't take this as 
a strong endorsement of doing the fix in 2.6; the set-an-attribute-to- 
fix-it idea is fine by me too.  I thought I'd lay out the reasoning for 
violating an otherwise extremely reasonable policy, though.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Barry Warsaw]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 4, 2009, at 2:44 PM, gl...@divmod.com wrote:

Maintaining compatibility with the 2.6.x version of asyncore  
presupposes that *someone* has written some software against that  
version of asyncore and it might break if they installed an upgrade,  
though.  If that's the case - if there's even one person who has  
written or run any asyncore software against the version in 2.6 -  
then I think maintaining bug-for-bug compatibility is important.


However, my guess - and I'm assuming that JP was thinking the same  
thing - is that literally nobody has done that, or even *would* ever  
do that, so there's no software anywhere in the world that would  
break.  asyncore is relatively unpopular (thanks in part to the  
excellent alternatives :-)); its major users are the same people who  
have already complained.


FWIW, I use smtpd.py and a few of the asyncore APIs  
(.loop(), .socket_map.clear(), and .close_all()) in the Mailman 3 test  
suite.  That only works on Python 2.6 and I don't recall even a hiccup  
when moving from 2.5 to 2.6.


Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSa7kiXEjvBPtnXfVAQI2BQP/XD71ipSv5OhtdOJt+OF2jbfOyFU/2vPR
iJ33g9JJ2A0CvDWXmc81t9hMH3U8waagkXaDUF4r2MNM3JbONSjQGGZqX6X/1UOB
XCT+jNOrbRw2VQR0qjrweQ5A8u0Y1zsDo/rD4CXe0K1pFFWLubaU3lVgqEBMBL8r
xvf77EMoUuA=
=vmM5
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Bill Janssen]

Guido van Rossum  wrote:

> On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone  
> wrote:
> > On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum 
> > wrote:
> >>
> >> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
> >> wrote:
> >> [snip]
> >>>
> >>> So, as a disinterested party in this specific case, I'd say revert to
> >>> the pre-2.6 behavior.  It does less harm than leaving the current
> >>> behavior.
> >>
> >> Sorry, but I really do think that we should maintain backward
> >> compatibility *within* the 2.6 series as well. If that makes it
> >> impossible to also maintain the 2.5 behavior, perhaps some flag could
> >> be added to restore 2.5 compatibility, e.g.
> >>
> >> import asyncore
> >> asyncore.python_25_compat = True
> >>
> >> Note that this "API" is designed to work in 2.5 as well. :-)
> 
> > But why?  The argument I made had the objective of minimizing developer
> > effort.  What's the objective of maintaining backward compatibility within
> > the 2.6 series in this case (sorry if it appeared earlier in this thread
> > and I missed it)?
> 
> The same as always. We don't change APIs in bugfix releases.

OK, seems reasonable.

But in this case, isn't the broken API the bug that's being fixed?  Do
we need a different way to fix broken APIs in bugfix releases?

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Daniel Stutzbach]

On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum  wrote:

> The same as always. We don't change APIs in bugfix releases.
>

This question is actually for the Zope folks and others who have had
problems with the 2.6 asyncore/asynchat:

Are any of the problems due to a change in the documented API... or are they
all due to changes in undocumented internals that your code relied on?

Myself, the only change that I noticed going from 2.5 to 2.6 was that the
undocumented signature of asynchat.__init__() changed from taking a
parameter called "conn" to one with the same meaning called "sock".

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Ivan Krstić]

Glyph,

On Mar 4, 2009, at 12:31 AM, gl...@divmod.com wrote:
If someone who has PEP-writing skills and some free time is updating  
asyncore to be more modern, I'd be very happy to walk that person  
through Twisted's API design and explain how either portions of our  
implementation could be lifted for the stdlib, or how a "high level"  
layer could be written into asyncore so that the author of a  
particular chunk of low-level networking code could ignore whether  
they're using the stdlib mainloop or one of the Twisted ones.


I spent about a half hour sometime in the last month talking this  
through with Itamar, though not in great detail. I'd be interested in  
sitting down with both of you and trying to establish more precisely  
how much work is necessary to get something to actually happen here. I  
won't outright promise a PEP, but I'll promise at the very least to  
write down elaborate notes that someone could turn into a PEP  
relatively straightforwardly. Deal?


Cheers,

--
Ivan Krstić  | http://radian.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Chris McDonough]

Daniel Stutzbach wrote:
> On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum  wrote:
> 
>> The same as always. We don't change APIs in bugfix releases.
>>
> 
> This question is actually for the Zope folks and others who have had
> problems with the 2.6 asyncore/asynchat:
> 
> Are any of the problems due to a change in the documented API... or are they
> all due to changes in undocumented internals that your code relied on?

As far as I can tell, asyncore/asynchat is all "undocumented internals".  Any
use of asyncore in anger will use internals; there never was any well-understood
API to these modules.  Medusa itself (from which asyncore and asynchat were
derived) appears to itself break with the changes to asyncore/asynchat in 2.6
(at least it appears to use attributes like "ac_out_buffer" which were removed
in 2.6; this is not "Zope's version"; there is no such animal; this is plain old
Medusa 0.5.4).

Count me in as one who believes that it would be the lesser of two evils to
revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its
asyncore/asynchat more forward compatible with 3.0's than backwards compatible
with 2.5's and prior; this was just a mistake and keeping old code running
should trump any theoretical or puritanical benefit in a dot release.

- C

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough  wrote:
> Daniel Stutzbach wrote:
>> On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum  wrote:
>>
>>> The same as always. We don't change APIs in bugfix releases.
>>>
>>
>> This question is actually for the Zope folks and others who have had
>> problems with the 2.6 asyncore/asynchat:
>>
>> Are any of the problems due to a change in the documented API... or are they
>> all due to changes in undocumented internals that your code relied on?
>
> As far as I can tell, asyncore/asynchat is all "undocumented internals".  Any
> use of asyncore in anger will use internals; there never was any 
> well-understood
> API to these modules.  Medusa itself (from which asyncore and asynchat were
> derived) appears to itself break with the changes to asyncore/asynchat in 2.6
> (at least it appears to use attributes like "ac_out_buffer" which were removed
> in 2.6; this is not "Zope's version"; there is no such animal; this is plain 
> old
> Medusa 0.5.4).
>
> Count me in as one who believes that it would be the lesser of two evils to
> revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
> rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
> certainly has fewer, if any, consumers); somebody messed up in 2.6 by making 
> its
> asyncore/asynchat more forward compatible with 3.0's than backwards compatible
> with 2.5's and prior; this was just a mistake and keeping old code running
> should trump any theoretical or puritanical benefit in a dot release.

Then I'd like to hear from the folks who made and reviewed those
changes to asyncore in 2.6. I can't imagine the changes were done
without good intentions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Daniel Stutzbach]

On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić <
krs...@solarsail.hcs.harvard.edu> wrote:

> I spent about a half hour sometime in the last month talking this through
> with Itamar, though not in great detail. I'd be interested in sitting down
> with both of you and trying to establish more precisely how much work is
> necessary to get something to actually happen here. I won't outright promise
> a PEP, but I'll promise at the very least to write down elaborate notes that
> someone could turn into a PEP relatively straightforwardly. Deal?
>

Will any or all of you be at PyCon?  I'd be willing to put in the extra work
to turn your notes into a PEP.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Steve Holden]

Daniel Stutzbach wrote:
> On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić
>  > wrote:
> 
> I spent about a half hour sometime in the last month talking this
> through with Itamar, though not in great detail. I'd be interested
> in sitting down with both of you and trying to establish more
> precisely how much work is necessary to get something to actually
> happen here. I won't outright promise a PEP, but I'll promise at the
> very least to write down elaborate notes that someone could turn
> into a PEP relatively straightforwardly. Deal?
> 
> 
> Will any or all of you be at PyCon?  I'd be willing to put in the extra
> work to turn your notes into a PEP.
> 
OPEN SPACE!
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Wed, Mar 4, 2009 at 1:31 PM, Guido van Rossum  wrote:
> On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough  wrote:
>> Daniel Stutzbach wrote:
>>> On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum  wrote:
>>>
 The same as always. We don't change APIs in bugfix releases.

>>>
>>> This question is actually for the Zope folks and others who have had
>>> problems with the 2.6 asyncore/asynchat:
>>>
>>> Are any of the problems due to a change in the documented API... or are they
>>> all due to changes in undocumented internals that your code relied on?
>>
>> As far as I can tell, asyncore/asynchat is all "undocumented internals".  Any
>> use of asyncore in anger will use internals; there never was any 
>> well-understood
>> API to these modules.  Medusa itself (from which asyncore and asynchat were
>> derived) appears to itself break with the changes to asyncore/asynchat in 2.6
>> (at least it appears to use attributes like "ac_out_buffer" which were 
>> removed
>> in 2.6; this is not "Zope's version"; there is no such animal; this is plain 
>> old
>> Medusa 0.5.4).
>>
>> Count me in as one who believes that it would be the lesser of two evils to
>> revert to the older (2.5 and prior) asyncore/asynchat implementations in 
>> 2.6.2
>> rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
>> certainly has fewer, if any, consumers); somebody messed up in 2.6 by making 
>> its
>> asyncore/asynchat more forward compatible with 3.0's than backwards 
>> compatible
>> with 2.5's and prior; this was just a mistake and keeping old code running
>> should trump any theoretical or puritanical benefit in a dot release.
>
> Then I'd like to hear from the folks who made and reviewed those
> changes to asyncore in 2.6. I can't imagine the changes were done
> without good intentions.

Interestingly enough, I just accidentally found this thread with an
unrelated web search, but it makes sense to reply.

Why the changes to asyncore:
1) the internals of asyncore were inefficient
2) the API and general behavior of the handle_*(), readable() and
writable() methods are unchanged, which are *exactly* what a user
should be overriding with subclassing
3) anyone modifying the *internals* of the implementation for the sake
of speed, shouldn't need to anymore

Maybe I have misread the posts in this thread, but does anyone have
any examples of their code breaking when only overriding handle_*(),
readable(), and writable() methods?

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 4 Mar, 09:14 pm, krs...@solarsail.hcs.harvard.edu wrote:
I spent about a half hour sometime in the last month talking this 
through with Itamar, though not in great detail. I'd be interested in 
sitting down with both of you and trying to establish more precisely 
how much work is necessary to get something to actually happen here. I 
won't outright promise a PEP, but I'll promise at the very least to 
write down elaborate notes that someone could turn into a PEP 
relatively straightforwardly. Deal?


Absolutely.  I really appreciate the offer.  As the other gentlemen 
suggested, PyCon would be an ideal venue for doing this.  Are you going 
to be there?


I'll hopefully be seeing your talk this evening, but I suspect that 
would be a pretty bad venue to try to get this done ;-).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 4 Mar, 08:28 pm, ba...@python.org wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 4, 2009, at 2:44 PM, gl...@divmod.com wrote:
Maintaining compatibility with the 2.6.x version of asyncore 
presupposes that *someone* has written some software against that 
version of asyncore and it might break if they installed an upgrade, 
though.


FWIW, I use smtpd.py and a few of the asyncore APIs  (.loop(), 
.socket_map.clear(), and .close_all()) in the Mailman 3 test  suite. 
That only works on Python 2.6 and I don't recall even a hiccup  when 
moving from 2.5 to 2.6.


Well, in that case, I withdraw even my disinterested suggestion :).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 4 Mar, 09:36 pm, dan...@stutzbachenterprises.com wrote:
Will any or all of you be at PyCon?  I'd be willing to put in the extra 
work

to turn your notes into a PEP.


I definitely will be.  I'll see you there!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Neil Schemenauer]

Chris McDonough  wrote:
> As far as I can tell, asyncore/asynchat is all "undocumented
> internals".  Any use of asyncore in anger will use internals;
> there never was any well-understood API to these modules.

What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc) 

* a select() or epoll like interface that allows waiting on
  multiple channels

The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.

  Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Daniel Stutzbach]

On Thu, Mar 5, 2009 at 1:30 PM, Neil Schemenauer  wrote:

> What I would like to see is a module that provides a low-level API
> for doing cross-platform asynchronous IO.  The two necessary parts
> are:
>
>* a wrapper that allows non-blocking reads and writes on
>  channels (sockets, file descriptors, serial ports, etc)
>
>* a select() or epoll like interface that allows waiting on
>  multiple channels
>

Two thoughts:

If you have a working select(), it will tell you the sockets on which read()
and write() won't block, so non-blocking reads and writes are not necessary.

On Windows, sockets, pipes, and files are each completely distinct types
with their own functions and unifying them under one select()-like interface
requires faking it using threads behind-the-scenes AFAIK.  Personally, I'd
be happy with continuing to only support socket objects on Windows.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Neil Schemenauer wrote:


What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc) 


* a select() or epoll like interface that allows waiting on
  multiple channels


+1

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Daniel Stutzbach wrote:

If you have a working select(), it will tell you the sockets on which 
read() and write() won't block, so non-blocking reads and writes are not 
necessary.


No, but there should be an interface that lets you say
"when something comes in on this fd, call this function
for me".

In other words it should be a light wrapper around
select/poll/whatever that provides a callback interface.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing  wrote:
> Daniel Stutzbach wrote:
>
>> If you have a working select(), it will tell you the sockets on which
>> read() and write() won't block, so non-blocking reads and writes are not
>> necessary.
>
> No, but there should be an interface that lets you say
> "when something comes in on this fd, call this function
> for me".
>
> In other words it should be a light wrapper around
> select/poll/whatever that provides a callback interface.

A read callback, a write callback.  What about close, error, connect,
and accept callbacks?

I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Bill Janssen]

Josiah Carlson  wrote:

> On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing  
> wrote:
> > Daniel Stutzbach wrote:
> >
> >> If you have a working select(), it will tell you the sockets on which
> >> read() and write() won't block, so non-blocking reads and writes are not
> >> necessary.
> >
> > No, but there should be an interface that lets you say
> > "when something comes in on this fd, call this function
> > for me".
> >
> > In other words it should be a light wrapper around
> > select/poll/whatever that provides a callback interface.
> 
> A read callback, a write callback.  What about close, error, connect,
> and accept callbacks?
> 
> I hate to say it (not really), but that's pretty much the handle_*()
> methods of asyncore :/ .

What asyncore was missing was a timer API (a way to register functions
to be called periodically).  Then it would be pretty much like any other
event loop system.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Josiah Carlson wrote:


A read callback, a write callback.  What about close, error, connect,
and accept callbacks?


Yep, all those as well.


I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .


Well, then, what's the problem?

Is there anything else people want that asyncore doesn't
provide, or is it just a matter of nailing down the existing
API a little?

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Thu, Mar 5, 2009 at 1:09 PM, Bill Janssen  wrote:
> Josiah Carlson  wrote:
>
>> On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing  
>> wrote:
>> > Daniel Stutzbach wrote:
>> >
>> >> If you have a working select(), it will tell you the sockets on which
>> >> read() and write() won't block, so non-blocking reads and writes are not
>> >> necessary.
>> >
>> > No, but there should be an interface that lets you say
>> > "when something comes in on this fd, call this function
>> > for me".
>> >
>> > In other words it should be a light wrapper around
>> > select/poll/whatever that provides a callback interface.
>>
>> A read callback, a write callback.  What about close, error, connect,
>> and accept callbacks?
>>
>> I hate to say it (not really), but that's pretty much the handle_*()
>> methods of asyncore :/ .
>
> What asyncore was missing was a timer API (a way to register functions
> to be called periodically).  Then it would be pretty much like any other
> event loop system.

There are two variants of patches to offer timer API functionality in
the bug tracker right now.  One from Giampaolo that uses a variant of
Twisted's scheduler, one from me that uses an updated sched.py .
Giampaolo's is more complete (it has documentation and tests), but
mine is more efficient with nontrivial task lists.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Antoine Pitrou]

Greg Ewing  canterbury.ac.nz> writes:
> 
> Neil Schemenauer wrote:
> 
> > What I would like to see is a module that provides a low-level API
> > for doing cross-platform asynchronous IO.  The two necessary parts
> > are:
> > 
> > * a wrapper that allows non-blocking reads and writes on
> >   channels (sockets, file descriptors, serial ports, etc) 

For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?
(I'm only suggesting it here, I don't intend to do this myself)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 07:30 pm, n...@arctrix.com wrote:

Chris McDonough  wrote:

As far as I can tell, asyncore/asynchat is all "undocumented
internals".  Any use of asyncore in anger will use internals;
there never was any well-understood API to these modules.



The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.


... which is exactly why I have volunteered to explain to someone how to 
separate the core event-loop bits (suitable for inclusion in the 
standard library) from the huge pile of protocol implementations which 
are not necessarily useful.


Despite the FUD to the contrary, Twisted's internal factoring is quite 
good; it's not a single, undifferentiated pile of code.  Plus, at this 
point, we're not even talking about actually putting any Twisted code 
into the standard library, just standardizing the "protocol" API, which 
basically boils down to:


 connectionMade() -> your connection has begun
 dataReceived(data) -> you got some bytes, handle them
 connectionLost(reason) -> your connection has gone away (with an object 
explaining why)


and the inverse, "transport", which is:

 write(data) -> deliver some data to the dataReceived on the other end 
of this connection (non-blocking, with buffering)

 loseConnection() -> goodbye

There are a few other minor details related to how you set these up to 
talk to each other and tell when the out-buffer is empty, but it's all 
pretty straightforward.  The main point is that you don't ever call 
recv() or send() and deal with buffering or handling weird errno values. 
For example, if your connection goes away, the notification you get is 
"your connection went away", not "oops you tried to read some bytes, but 
your connection was gone by the time you tried, even though I just told 
you it was ready for reading" or other similarly obtuse failure modes.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Thu, Mar 5, 2009 at 3:11 PM,   wrote:
>
> On 07:30 pm, n...@arctrix.com wrote:
>>
>> Chris McDonough  wrote:
>>>
>>> As far as I can tell, asyncore/asynchat is all "undocumented
>>> internals".  Any use of asyncore in anger will use internals;
>>> there never was any well-understood API to these modules.
>
>> The implementation requires some intricate and platform specific
>> code which is why it would be nice to be a standard library feature.
>>
>> I'm sure that Twisted has the necessary parts but the problem, IMHO,
>> is that it does so much more else.
>
> ... which is exactly why I have volunteered to explain to someone how to
> separate the core event-loop bits (suitable for inclusion in the standard
> library) from the huge pile of protocol implementations which are not
> necessarily useful.
>
> Despite the FUD to the contrary, Twisted's internal factoring is quite good;
> it's not a single, undifferentiated pile of code.  Plus, at this point,
> we're not even talking about actually putting any Twisted code into the
> standard library, just standardizing the "protocol" API, which basically
> boils down to:
>
>  connectionMade() -> your connection has begun
>  dataReceived(data) -> you got some bytes, handle them
>  connectionLost(reason) -> your connection has gone away (with an object
> explaining why)
>
> and the inverse, "transport", which is:
>
>  write(data) -> deliver some data to the dataReceived on the other end of
> this connection (non-blocking, with buffering)
>  loseConnection() -> goodbye
>
> There are a few other minor details related to how you set these up to talk
> to each other and tell when the out-buffer is empty, but it's all pretty
> straightforward.  The main point is that you don't ever call recv() or
> send() and deal with buffering or handling weird errno values. For example,
> if your connection goes away, the notification you get is "your connection
> went away", not "oops you tried to read some bytes, but your connection was
> gone by the time you tried, even though I just told you it was ready for
> reading" or other similarly obtuse failure modes.

Those "obtuse failure modes" are handled fairly well by asynchat.  It
works pretty well, and includes sample implementations to get you 99%
of the way towards the 5-method API you describe (which is why a lot
of people use asynchat instead of asyncore).  If it weren't for the
fact that asynchat had a previously existing API (including the
.push(), collect_incoming_data(), and found_terminator()), I think it
could stand for methods to make it fit better with asyncore's
handle_*() methods.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Antoine Pitrou wrote:


For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?


I think we need to be clearer what we mean when we talk
about non-blocking in this context. Normally when you're
using select/poll you *don't* make the underlying file
descriptor non-blocking in the OS sense. The non-blockingness
comes from the fact that you're using select/poll to make
sure the fd is ready before you access it.

So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Hrvoje Niksic]

Greg Ewing wrote:

Antoine Pitrou wrote:


For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?


I think we need to be clearer what we mean when we talk
about non-blocking in this context. Normally when you're
using select/poll you *don't* make the underlying file
descriptor non-blocking in the OS sense. The non-blockingness
comes from the fact that you're using select/poll to make
sure the fd is ready before you access it.

So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.


This is not necessarily the case.  In fact, modern sources often 
recommend using poll along with the non-blocking sockets for (among 
other things) performance reasons.  For example, when a non-blocking 
socket becomes readable, you don't read from it only once and go back to 
the event loop, you read from it in a loop until you get EAGAIN.  This 
allows for processing of fast-incoming data with fewer system calls.


Linux's select(2) man page includes a similar advice with different 
motivation:


   Under Linux, select() may report a socket file descriptor
   as "ready for reading",  while  nevertheless
   a subsequent read blocks.  This could for example
   happen when data has arrived but upon
   examination has wrong checksum and is discarded.  There may
   be other circumstances  in  which  a
   file  descriptor  is  spuriously  reported  as ready.
   Thus it may be safer to use O_NONBLOCK on
   sockets that should not block.

Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Hrvoje Niksic wrote:


   Under Linux, select() may report a socket file descriptor
   as "ready for reading",  while  nevertheless
   a subsequent read blocks.


Blarg. Linux is broken, then. This should not happen.


   This could for example
   happen when data has arrived but upon
   examination has wrong checksum and is discarded.


That's no excuse -- the kernel should check all its
checksums *before* waking up selecting processes!

Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.


I'm not saying there isn't merit in having support for
non-blocking file descriptors, only that it's not in
any sense a prerequisite or first step towards a
select/poll wrapper. They're orthogonal issues, even
if you might sometimes want to use them together.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Hrvoje Niksic]

Greg Ewing wrote:
Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.


I'm not saying there isn't merit in having support for
non-blocking file descriptors, only that it's not in
any sense a prerequisite or first step towards a
select/poll wrapper. They're orthogonal issues, even
if you might sometimes want to use them together.


In that case we are in agreement.  Looking back, I was somewhat confused 
by this paragraph:


So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.

If they're orthogonal, then it does make sense to talk about having a 
separate non-blocking socket API and poll API, even if the latter can be 
used to implement non-blocking *functionality* (hypothetical Linux 
issues aside).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 10:01 am, greg.ew...@canterbury.ac.nz wrote:

Hrvoje Niksic wrote:

   Under Linux, select() may report a socket file descriptor
   as "ready for reading",  while  nevertheless
   a subsequent read blocks.


Blarg. Linux is broken, then. This should not happen.


You know what else is broken?  MacOS, FreeBSD, Solaris, and every 
version of Windows.  I haven't tried running Twisted on AmigaOS but I 
bet it has some problems too.


On occasion Linux has been so badly broken that Twisted has motivated a 
fix.  For example, 
http://lkml.indiana.edu/hypermail/linux/kernel/0502.3/1160.html


But even if we ignore difficulties at the OS level (which should, after 
all, be worked around rather than catered to in API design) there are 
other good reasons why the general async API should be fairly distant 
from both the select/poll wrapper and the questions of blocking vs. non- 
blocking sockets.  For another example, consider the issue of non- 
blocking SSL sockets.  Sometimes, in order to do a "read", you actually 
need to do a write and then another read.  Which means that application 
code, if it wants to be compatible with SSL, needs to deal with any 
failure that may come from a write as coming from a read, unless you 
abstract all this away somehow.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Neil Schemenauer]

gl...@divmod.com  wrote:
> ... which is exactly why I have volunteered to explain to someone how to 
> separate the core event-loop bits (suitable for inclusion in the 
> standard library) from the huge pile of protocol implementations which 
> are not necessarily useful.
>
> Despite the FUD to the contrary, Twisted's internal factoring is quite 
> good; it's not a single, undifferentiated pile of code.  Plus, at this 
> point, we're not even talking about actually putting any Twisted code 
> into the standard library, just standardizing the "protocol" API, which 
> basically boils down to:
[...]

This sounds great.  I'm interested on working on this since it
scratches an itch of mine but I don't know if I will have time.  Do
you think if this part of Twisted became part of the standard
library that it would be used by Twisted or would it continue to
have its own version?

  Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[skip]

Glyph> ... which is exactly why I have volunteered to explain to someone
Glyph> how to separate the core event-loop bits (suitable for inclusion
Glyph> in the standard library) from the huge pile of protocol
Glyph> implementations which are not necessarily useful.

Neil> This sounds great.  I'm interested on working on this since it
Neil> scratches an itch of mine but I don't know if I will have time.
Neil> Do you think if this part of Twisted became part of the standard
Neil> library that it would be used by Twisted or would it continue to
Neil> have its own version?

Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days sprinting,
one on Mailman/SpamBayes integration and one on Python core stuff.  This
might fit well into my Python core "slot".  I will probably have little time
before the sprint to do much, but any brain dumps or Twisted pointers people
could give me in the interim would be appreciated.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 02:28 am, s...@pobox.com wrote:
   Glyph> ... which is exactly why I have volunteered to explain to 
someone

   Glyph> how to separate the core event-loop bits



   Neil> This sounds great.



Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days 
sprinting,
one on Mailman/SpamBayes integration and one on Python core stuff. 
This
might fit well into my Python core "slot".  I will probably have little 
time
before the sprint to do much, but any brain dumps or Twisted pointers 
people

could give me in the interim would be appreciated.


I'll try to make sure we get those notes to you in advance of the 
sprints :).  I should be at the Twisted sprint the whole time - will you 
be at the physical sprint, or following along at home?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 01:50 am, n...@arctrix.com wrote:

gl...@divmod.com  wrote:



we're not even talking about actually putting any Twisted code
into the standard library, just standardizing the "protocol" API, 
which

basically boils down to:

[...]

This sounds great.  I'm interested on working on this since it
scratches an itch of mine but I don't know if I will have time.  Do
you think if this part of Twisted became part of the standard
library that it would be used by Twisted or would it continue to
have its own version?


As I said above, right now I'm just talking about the interface.  There 
are several implementations of the main loop that call those interfaces, 
as well as test implementations.


Of course I hope we can eliminate some redundancy, but that's step 2: 
before we refactor, we have to make the code that we're refactoring 
actually repetitive rather than just similar.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[skip]

Glyph> I'll try to make sure we get those notes to you in advance of the
Glyph> sprints :).  I should be at the Twisted sprint the whole time -
Glyph> will you be at the physical sprint, or following along at home?

Thanks.  I will be at the physical sprints.  (I live in the Chicago area.)

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Daniel Stutzbach]

On Sat, Mar 7, 2009 at 8:28 PM,  wrote:

> Anybody interested in working on this at a PyCon Sprint?  I won't be
> attending the conference proper, but plan to spend a couple days sprinting,
>

I'll be there and interested. :)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 02:46 pm, dan...@stutzbachenterprises.com wrote:

On Sat, Mar 7, 2009 at 8:28 PM,  wrote:

Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days 
sprinting,



I'll be there and interested. :)


Great!  I plan to hold both of you to that :).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Bill Janssen]

Sidnei da Silva <[EMAIL PROTECTED]> wrote:

> https://bugs.edge.launchpad.net/zope2/+bug/280020

I think there are real issues here with both asynchat and Medusa.
Asynchat has been heavily re-written, and the "ac_out_buffer" has
apparently disappeared.  But "ac_out_buffer_size" is still there.  That
strikes me as odd, and probably means that asynchat.py needs more
changes.  However, Medusa (basically just an application layer on top
of asyncore/asynchat) also needs to be re-written to take account of
the changes in asynchat.

Bill

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Wed, Oct 8, 2008 at 10:30 AM, Bill Janssen <[EMAIL PROTECTED]> wrote:
> Sidnei da Silva <[EMAIL PROTECTED]> wrote:
>
>> https://bugs.edge.launchpad.net/zope2/+bug/280020
>
> I think there are real issues here with both asynchat and Medusa.
> Asynchat has been heavily re-written, and the "ac_out_buffer" has
> apparently disappeared.  But "ac_out_buffer_size" is still there.  That
> strikes me as odd, and probably means that asynchat.py needs more
> changes.  However, Medusa (basically just an application layer on top
> of asyncore/asynchat) also needs to be re-written to take account of
> the changes in asynchat.

ac_out_buffer was removed because it is unneeded (we have a deque; why
rely on an extra attribute?).
ac_out_buffer_size still remains as a blocksize in which to pre-split
outgoing data (if you have a 100k string you want to send, repeatedly
slicing it as you are able to send pieces is slow, but pre-slicing
chunks is fast, and generally results in being able to pass full chunk
to the underlying TCP/IP stack).

But yes, zope needs to be changed to reflect the updated
asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
easier to use now.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Sidnei da Silva]

On Wed, Oct 8, 2008 at 6:26 PM, Josiah Carlson <[EMAIL PROTECTED]> wrote:
> ac_out_buffer was removed because it is unneeded (we have a deque; why
> rely on an extra attribute?).
> ac_out_buffer_size still remains as a blocksize in which to pre-split
> outgoing data (if you have a 100k string you want to send, repeatedly
> slicing it as you are able to send pieces is slow, but pre-slicing
> chunks is fast, and generally results in being able to pass full chunk
> to the underlying TCP/IP stack).
>
> But yes, zope needs to be changed to reflect the updated
> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
> easier to use now.

Great to know. So medusa should be changed similarly to the changes
made to asynchat? Your suggestion on the bugtracker is even better: to
subclass asynchat as much as possible.

-- 
Sidnei da Silva
Enfold Systemshttp://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Bill Janssen]

Sidnei da Silva <[EMAIL PROTECTED]> wrote:

> Great to know. So medusa should be changed similarly to the changes
> made to asynchat?

Hmmm.  Been a long time since a Medusa release, but I need a working
Medusa, too.

> Your suggestion on the bugtracker is even better: to
> subclass asynchat as much as possible.

That's basically what Medusa is, a set of subclasses.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Bill Janssen]

Josiah Carlson <[EMAIL PROTECTED]> wrote:

> But yes, zope needs to be changed to reflect the updated
> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
> easier to use now.

Just for completeness, I built a fresh 2.6, installed Medusa (from
http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
as well as it does on 2.5, anyway).  I think this is a Zope issue.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Guido van Rossum]

On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <[EMAIL PROTECTED]> wrote:
> Josiah Carlson <[EMAIL PROTECTED]> wrote:
>
>> But yes, zope needs to be changed to reflect the updated
>> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
>> easier to use now.
>
> Just for completeness, I built a fresh 2.6, installed Medusa (from
> http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
> as well as it does on 2.5, anyway).  I think this is a Zope issue.

Way back in the day, Zope monkeypatched whole parts of asyncore,
replacing them with some of its own code. If that's still the case,
this breakage should be no surprise.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Guido van Rossum wrote:
> On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <[EMAIL PROTECTED]> wrote:
>> Josiah Carlson <[EMAIL PROTECTED]> wrote:
>>
>>> But yes, zope needs to be changed to reflect the updated
>>> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
>>> easier to use now.
>> Just for completeness, I built a fresh 2.6, installed Medusa (from
>> http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
>> as well as it does on 2.5, anyway).  I think this is a Zope issue.
> 
> Way back in the day, Zope monkeypatched whole parts of asyncore,
> replacing them with some of its own code. If that's still the case,
> this breakage should be no surprise.

Zope has been hooking the 'asyncore.loop' function (wrapping it in order
to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is
the earliest checkout I have, and it was branched in early January 2002).

Zope also began patched asyncore's logging functions in 2.7, and later
updated that patch to make the logger use the stdlib 'logging' module.

I think the change we need to make (in ZServer/medusa/http_server.py) is
to emulate the new 'writeable' implementation in asynchat, at least when
running under 2.6.

Zope's 'ZServer.medusa.http_server.http_request.writable()'
implementation is::

def writable (self):
# this is just the normal async_chat 'writable',
# here for comparison
return self.ac_out_buffer or len(self.producer_fifo)


The Python 2.5 asynchat.asynchat.writable does::

def writable (self):
"predicate for inclusion in the writable for select()"
# return len(self.ac_out_buffer) or len(self.producer_fifo) or
# (not self.connected)
# this is about twice as fast, though not as clear.
return not (
(self.ac_out_buffer == '') and
self.producer_fifo.is_empty() and
self.connected
)

The Python 2.6 asynchat.asynchat.writable now does::

def writable (self):
"predicate for inclusion in the writable for select()"
return self.producer_fifo or (not self.connected)


medusa 0.5.4's medusa.http_server.http_request doesn't override
'writable', but it does have a broken 'writeable_for_proxy':


def writable_for_proxy (self):
# this version of writable supports the idea of a 'stalled'
# producer
# [i.e., it's not ready to produce any output yet] This is
# needed by
# the proxy, which will be waiting for the magic combination of
# 1) hostname resolved
# 2) connection made
# 3) data available.
if self.ac_out_buffer:
return 1
elif len(self.producer_fifo):
p = self.producer_fifo.first()
if hasattr (p, 'stalled'):
return not p.stalled()
else:
return 1


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFI7ix3+gerLs4ltQ4RAldnAKC/QLJHmdE9dxInkWuIGja0gtSXYwCcCJcH
6XooEwW/AkJ1ntmGyxi8urM=
=1kps
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com