Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Antoine Pitrou wrote:


For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?


I think we need to be clearer what we mean when we talk
about non-blocking in this context. Normally when you're
using select/poll you *don't* make the underlying file
descriptor non-blocking in the OS sense. The non-blockingness
comes from the fact that you're using select/poll to make
sure the fd is ready before you access it.

So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pickler/Unpickler API clarification

[Greg Ewing]

Guido van Rossum wrote:


Then it'd be better to have a method clear_memo() on pickle objects.


You should have that anyway. I was just suggesting a
way of preserving compatibility with old code without
exposing all the details of the memo.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] patch commit policies (was [issue4308] repr of httplib.IncompleteRead is stupid)

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Chris Withers wrote:
> Martin v. Löwis wrote:
>> Martin v. Löwis  added the comment:
>>
>>> So all Chris has to do to get this applied to 2.5 is craft an exploit based
>>> on the current behavior, right? ;-)
>> Right :-) Of course, security patches should see a much more careful
>> review than regular bug fixes.
> 
> Well, it's funny you say that, since where I bumped into this, the bug 
> was effectively DOS'ing a couple of mailservers as a result of 
> mailinglogger sending out log entries of uncaught exceptions such as 
> this and so emitting 100Mb emails whenever the foreign server chose not 
> to deliver the whole chunk requested...

If it is possible for a hostile outsider to trigger the DOS by sending
mail to be processed by an application using the library, and the
application can't avoid the DOS without ditching / forking /
monkeypatching the library, then I would call the bug a "security bug",
period.

As for backward compatibility:  any application which is depending on
getting arbitrarily-long lines in its logfile is already insane, and
should be scrapped.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJsJOB+gerLs4ltQ4RAva/AKC2Ta0edNMxMLxXQM6+WsB4AKo10QCdFF58
ghfy8pT6VlrO0z0QoXnjL7o=
=9lCT
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] html5lib/BeautifulSoup (was: Integrate lxml into the stdlib? (was: Integrate BeautifulSoup into stdlib?))

[Jim Jewett]

Stefan Behnel wrote:

> I would have a hard time feeling happy
> if a real-world HTML parser was added to the stdlib that provides a totally
> different interface than the best (and fastest) XML library that the stdlib
> currently has.

I doubt there would be any objection to someone contributing wrappers
for upgrades, but I wouldn't count on them being used.

lxml may well be the best choice for xml.

BeautifulSoup and html5lib wouldn't even exist if that actually
mattered for most of *their* use cases.  Think of them more as
pre-processors, like tidylib.  If enough web pages were even valid
HTML (let alone valid and well-formed XML), no one would have bothered
to write these libraries.

BeautifulSoup has the advantage of being long-proven in practice, for
ugly html.  (You mention an lxml feature with a similar intent, but
for lxml, it is one of several addon features; for BeautifulSoup, this
is the whole point.)

html5lib does not have as long of a history, but it does have the
advantage of being almost an endorsed standard.  Much of HTML 5 is
documenting the workarounds that browser makers already actually
employ to handle erroneous input, so that the complexities can at
least stop compounding.  html5lib is intended as a reference
implementation, and the w3c editor has used it to motivate changes in
the specification draft.  (This may make it unsuitable for inclusion
in the stdlib today, because of timing issues.)  In other words, it
isn't just the heuristics of one particular development team; it is
(modulo bugs, and after official publication) the heuristics that the
major web browser makers have agreed to treat as "correct" in the
future.

-jJ
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] patch commit policies (was [issue4308] repr of httplib.IncompleteRead is stupid)

[Barry Warsaw]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 5, 2009, at 6:18 PM, Martin v. Löwis wrote:


That aside, is it actually a python-wide policy to *forbid* patching
older releases where the patch isn't security-related?


I set this policy for the releases I manage, namely 2.4 and 2.5.


This is a Python-wide policy.

When Python 2.7 is released, there will be one last 2.6.x bug fix  
release, and then it will go into security-only mode.  Similarly, when  
Python 3.1 is released, there will be one last 3.0.x release and it  
too will go into security-only mode.


Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbBgJHEjvBPtnXfVAQIEbQP6Ah9I4SikMQ++vlSYb4BdNgw0VXF/8PMH
aTkG0+haOBoJ+mJ9G5GbBiJVtnV0B6qRV1gSV5FSIOzbQE/t0zU27APwnIWv57l9
uZIyyjU1AA0Gz5DSOsC9xtoAqEu/iFH9aRd17tE/N2Yib5p62myqAUVHIRPBl0fD
sh+ztoL3q2c=
=vgz0
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib?

[Martin v. Löwis]

> I do see the point you are making here. Even if lxml gets mature and
> static, that doesn't necessarily apply to the external libraries it uses.
> However, I should note that exactly the same argument also applies to
> sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3 being
> a fairly recent addition.

Fortunately, it is possible for users to just replace the sqlite DLL in
a Python installation, with no need of recompiling anything.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] patch commit policies (was [issue4308] repr of httplib.IncompleteRead is stupid)

[Martin v. Löwis]

> That aside, is it actually a python-wide policy to *forbid* patching
> older releases where the patch isn't security-related?

I set this policy for the releases I manage, namely 2.4 and 2.5.

I still plan to write a PEP on security releases, and how they relate
to maintenance releases.

> I can understand the "no more releases unless there are security
> problems", but what's the harm in applying a patch to an old version
> branch on the off chance that a security release might be made some time?

Yes. *Every* change causes the risk of breaking something. In fact, for
any non-doc change, it is possible to construct a program that breaks
under the change.

The longer a release is in production use, the less breakage can be
risked. People will have worked around all regular bugs that they may
have run into. So when they ever make the experience that installing
a security fix actually breaks their working code, they will refrain
from ever installing Python patches again.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Thu, Mar 5, 2009 at 3:11 PM,   wrote:
>
> On 07:30 pm, n...@arctrix.com wrote:
>>
>> Chris McDonough  wrote:
>>>
>>> As far as I can tell, asyncore/asynchat is all "undocumented
>>> internals".  Any use of asyncore in anger will use internals;
>>> there never was any well-understood API to these modules.
>
>> The implementation requires some intricate and platform specific
>> code which is why it would be nice to be a standard library feature.
>>
>> I'm sure that Twisted has the necessary parts but the problem, IMHO,
>> is that it does so much more else.
>
> ... which is exactly why I have volunteered to explain to someone how to
> separate the core event-loop bits (suitable for inclusion in the standard
> library) from the huge pile of protocol implementations which are not
> necessarily useful.
>
> Despite the FUD to the contrary, Twisted's internal factoring is quite good;
> it's not a single, undifferentiated pile of code.  Plus, at this point,
> we're not even talking about actually putting any Twisted code into the
> standard library, just standardizing the "protocol" API, which basically
> boils down to:
>
>  connectionMade() -> your connection has begun
>  dataReceived(data) -> you got some bytes, handle them
>  connectionLost(reason) -> your connection has gone away (with an object
> explaining why)
>
> and the inverse, "transport", which is:
>
>  write(data) -> deliver some data to the dataReceived on the other end of
> this connection (non-blocking, with buffering)
>  loseConnection() -> goodbye
>
> There are a few other minor details related to how you set these up to talk
> to each other and tell when the out-buffer is empty, but it's all pretty
> straightforward.  The main point is that you don't ever call recv() or
> send() and deal with buffering or handling weird errno values. For example,
> if your connection goes away, the notification you get is "your connection
> went away", not "oops you tried to read some bytes, but your connection was
> gone by the time you tried, even though I just told you it was ready for
> reading" or other similarly obtuse failure modes.

Those "obtuse failure modes" are handled fairly well by asynchat.  It
works pretty well, and includes sample implementations to get you 99%
of the way towards the 5-method API you describe (which is why a lot
of people use asynchat instead of asyncore).  If it weren't for the
fact that asynchat had a previously existing API (including the
.push(), collect_incoming_data(), and found_terminator()), I think it
could stand for methods to make it fit better with asyncore's
handle_*() methods.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 07:30 pm, n...@arctrix.com wrote:

Chris McDonough  wrote:

As far as I can tell, asyncore/asynchat is all "undocumented
internals".  Any use of asyncore in anger will use internals;
there never was any well-understood API to these modules.



The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.


... which is exactly why I have volunteered to explain to someone how to 
separate the core event-loop bits (suitable for inclusion in the 
standard library) from the huge pile of protocol implementations which 
are not necessarily useful.


Despite the FUD to the contrary, Twisted's internal factoring is quite 
good; it's not a single, undifferentiated pile of code.  Plus, at this 
point, we're not even talking about actually putting any Twisted code 
into the standard library, just standardizing the "protocol" API, which 
basically boils down to:


 connectionMade() -> your connection has begun
 dataReceived(data) -> you got some bytes, handle them
 connectionLost(reason) -> your connection has gone away (with an object 
explaining why)


and the inverse, "transport", which is:

 write(data) -> deliver some data to the dataReceived on the other end 
of this connection (non-blocking, with buffering)

 loseConnection() -> goodbye

There are a few other minor details related to how you set these up to 
talk to each other and tell when the out-buffer is empty, but it's all 
pretty straightforward.  The main point is that you don't ever call 
recv() or send() and deal with buffering or handling weird errno values. 
For example, if your connection goes away, the notification you get is 
"your connection went away", not "oops you tried to read some bytes, but 
your connection was gone by the time you tried, even though I just told 
you it was ready for reading" or other similarly obtuse failure modes.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Antoine Pitrou]

Greg Ewing  canterbury.ac.nz> writes:
> 
> Neil Schemenauer wrote:
> 
> > What I would like to see is a module that provides a low-level API
> > for doing cross-platform asynchronous IO.  The two necessary parts
> > are:
> > 
> > * a wrapper that allows non-blocking reads and writes on
> >   channels (sockets, file descriptors, serial ports, etc) 

For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?
(I'm only suggesting it here, I don't intend to do this myself)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pickler/Unpickler API clarification

[Guido van Rossum]

On Thu, Mar 5, 2009 at 1:36 PM, Greg Ewing  wrote:
> Guido van Rossum wrote:
>
>> This was a bad idea (*), and I'd be happy to ban it -- but we'd
>> probably have to bump the pickle protocol version in order to maintain
>> backwards compatibility.
>
> If you're talking about multiple calls to dump() on the same
> pickler, it might be a bad idea for a network connection, but
> I don't see anything wrong with using it on a file, and I find
> it useful to do so sometimes. Banning it would be excessive, IMO.

I don't think I was thinking of that when I first designed pickle but
the use case makes some sense. I still wish we could ban it or somehow
make it *not* the default behavior; the bug in the App Engine bug I
referenced before was introduced by an experienced developer who
wasn't aware of this behavior and was simply trying to avoid
unnecessarily creating a new pickler for each call.

>> The exposition is unintentional but for historic reasons we can't just
>> remove it. :-(
>
> A compromise might be to provide a memo attribute that returns
> a wrapper around the underlying cache -- maybe with only a
> clear() method if that's all you want to support.

Then it'd be better to have a method clear_memo() on pickle objects.

Perhaps we should do the opposite, and have a separate API for reuse
*without* clearing the memo? .dump_reusing_memo() and
.load_reusing_memo().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Thu, Mar 5, 2009 at 1:09 PM, Bill Janssen  wrote:
> Josiah Carlson  wrote:
>
>> On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing  
>> wrote:
>> > Daniel Stutzbach wrote:
>> >
>> >> If you have a working select(), it will tell you the sockets on which
>> >> read() and write() won't block, so non-blocking reads and writes are not
>> >> necessary.
>> >
>> > No, but there should be an interface that lets you say
>> > "when something comes in on this fd, call this function
>> > for me".
>> >
>> > In other words it should be a light wrapper around
>> > select/poll/whatever that provides a callback interface.
>>
>> A read callback, a write callback.  What about close, error, connect,
>> and accept callbacks?
>>
>> I hate to say it (not really), but that's pretty much the handle_*()
>> methods of asyncore :/ .
>
> What asyncore was missing was a timer API (a way to register functions
> to be called periodically).  Then it would be pretty much like any other
> event loop system.

There are two variants of patches to offer timer API functionality in
the bug tracker right now.  One from Giampaolo that uses a variant of
Twisted's scheduler, one from me that uses an updated sched.py .
Giampaolo's is more complete (it has documentation and tests), but
mine is more efficient with nontrivial task lists.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] draft 3.1 release schedule

[Martin v. Löwis]

> So what is the solution?

In the specific case, I don't know. I recall that somebody offered to
pick up the change. I really didn't mean to suggest that the patch
will remain unnoticed - it was just a warning that it *might* remain
unnoticed.

The more general issue is that of patches being unreviewed for a long
time, whether they have educational background or some other background
(say, cross-compilation, HP-UX support, etc).

>From time to time, people ask what they can do push a change into Python
that they really think is important. I once offered that people who
want a patch in Python really badly should review 10 other patches in
return, up to the point where they make a recommendation about the fate
of the patches. I was then talked into accepting just 5 such patches.
I have since withdrawn this offer, because
a) I was the only one making that offer in public, and
b) I was sometimes not really able to respond in a timely manner
   when the offer was invoked, because of overload.

So, for the more general issue, I don't have a solution, either.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pickler/Unpickler API clarification

[Greg Ewing]

Guido van Rossum wrote:


This was a bad idea (*), and I'd be happy to ban it -- but we'd
probably have to bump the pickle protocol version in order to maintain
backwards compatibility.


If you're talking about multiple calls to dump() on the same
pickler, it might be a bad idea for a network connection, but
I don't see anything wrong with using it on a file, and I find
it useful to do so sometimes. Banning it would be excessive, IMO.


The exposition is unintentional but for historic reasons we can't just
remove it. :-(


A compromise might be to provide a memo attribute that returns
a wrapper around the underlying cache -- maybe with only a
clear() method if that's all you want to support.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib?

[Stefan Behnel]

Hi,

Brett Cannon wrote:
> On Thu, Mar 5, 2009 at 12:52, Stefan Behnel wrote:
>> Benjamin Peterson wrote:
>>> it depends on Cython, which is wonderful normally, but maybe
>>> difficult to deal with in language evolution since we wouldn't have
>>> direct control over the C sources.
>> I see the point, although I think that this can be dealt with by
>>
>> a) using a specific, stable release version of Cython for a specific Python
>> release, so that this Cython version can be bug fixed if required (it's
>> implemented in Python, after all)
> 
> So including Cython source in the stdlib and then check in the generated C
> code?

Did I give the impression that a) was my preferred solution? ;)


>> b) adding Cython to the stdlib and building with that
> 
> That's an entirely separate discussion (for which my initial answer is to
> not consider it until it has stabilized to a 1.0 release).

Yes, that *is* an entirely separate discussion - for which my initial
answer is to consider it as soon as it is in a state where the compiler is
good enough to be useful and the language it compiles is stable enough to
be future proof. The language is almost Python, and the core syntax
extensions (compared to Python 2.6/3.0) haven't changed for a couple of
releases (except for the buffer syntax, which I personally don't consider
core but really nice to have).

The official goal for a 1.0 release is to compile Python programs, BTW. I
don't think the stdlib needs to wait for that.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Josiah Carlson wrote:


A read callback, a write callback.  What about close, error, connect,
and accept callbacks?


Yep, all those as well.


I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .


Well, then, what's the problem?

Is there anything else people want that asyncore doesn't
provide, or is it just a matter of nailing down the existing
API a little?

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib?

[Guido van Rossum]

Stefan,

I recommend that you give up pushing for lxml in the stdlib. There are
many complex factors to be weighed but in the balance I am not
comfortable with it, and continued argumentation is not going to
change that.

Sorry,

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pickler/Unpickler API clarification

[Greg Ewing]

Collin Winter wrote:


Reusing the Pickler without clearing the
memo will produce pickles that are, as best I can see, invalid


I'm not sure what you mean by "reusing the pickler" here,
and how it can produce an invalid pickle.

I think what the docs mean by it is continuing to pickle
objects to the same file, but in a logically separate
block that doesn't share any references with the previous
one, e.g.

   pickle obj1
   pickle obj2
   ---clear memo---
   pickle obj3

The whole thing is still a valid pickle containing 3 objects,
whether the memo is cleared at any point or not, and can
be unpickled using 3 corresponding unpickle calls to a
single unpickler.


1) Should Pickler/Unpickler objects automatically clear their memos
when dumping/loading?


If you mean should every call to Pickler.dump() or
Unpickler.load() clear the memo first, definitely *NOT*.
It's explicitly part of the specification that you can
make multiple calls to dump() to build up a single pickle
that shares state, as long as you unpickle it using a
corresponding number of load() calls.


2) Is memo an intentionally exposed, supported part of the
Pickler/Unpickler API, despite the lack of documentation and tests?


I think the 2.4 and later docs make it clear that it's
no longer considered part of the public API, if it ever
was.

If seeding the memo is considered a legitimate need, an
API could be provided for doing that.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pickler/Unpickler API clarification

[Guido van Rossum]

On Thu, Mar 5, 2009 at 12:07 PM, Collin Winter  wrote:
> I'm working on some performance patches for cPickle, and one of the
> bigger wins so far has been replacing the Pickler's memo dict with a
> custom hashtable (and hence removing memo's getters and setters). In
> looking over this, Jeffrey Yasskin commented that this would break
> anyone who was accessing the memo attribute.
>
> I've found a few examples of code using the memo attribute ([1], [2],
> [3]), and there are probably more out there, but the memo attribute
> doesn't look like part of the API to me. It's only documented in
> http://docs.python.org/library/pickle.html as "you used to need this
> before Python 2.3, but don't anymore". However: I don't believe you
> should ever need this attribute.
>
> The usages of memo I've seen break down into two camps: clearing the
> memo, and wanting to explicitly populate the memo with predefined
> values. Clearing the memo is recommended as part of reusing Pickler
> objects, but I can't fathom when you would want to reuse a Pickler
> *without* clearing the memo. Reusing the Pickler without clearing the
> memo will produce pickles that are, as best I can see, invalid -- at
> least, pickletools.dis() rejects this, which is the closest thing we
> have to a validator.

I can explain this, as I invented this behavior. The use case was to
have a long-lived session between a client and a server which were
communicating repeatedly using pickles. The idea was that values that
had been transferred once wouldn't have to be sent across the wire
again -- they could just be referenced.

This was a bad idea (*), and I'd be happy to ban it -- but we'd
probably have to bump the pickle protocol version in order to maintain
backwards compatibility.

> Explicitly setting memo values has the same
> problem: an easy, very brittle way to produce invalid data.

Agreed this is just preposterous. It was never part of the plan.

> So my questions are these:
> 1) Should Pickler/Unpickler objects automatically clear their memos
> when dumping/loading?

Alas, there could be backwards compatibility issues. Bumping the
protocol could mitigate this.

> 2) Is memo an intentionally exposed, supported part of the
> Pickler/Unpickler API, despite the lack of documentation and tests?

The exposition is unintentional but for historic reasons we can't just
remove it. :-(

> Thanks,
> Collin
>
> [1] - 
> http://google.com/codesearch/p?hl=en#Qx8E-7HUBTk/trunk/google/appengine/api/memcache/__init__.py&q=lang:py%20%5C.memo
> [2] - 
> http://google.com/codesearch/p?hl=en#M-DDI-lCOgE/lib/python2.4/site-packages/cvs2svn_lib/primed_pickle.py&q=lang:py%20%5C.memo
> [3] - 
> http://google.com/codesearch/p?hl=en#l_w_cA4dKMY/AtlasAnalysis/2.0.3-LST-1/PhysicsAnalysis/PyAnalysis/PyAnalysisUtils/python/root_pickle.py&q=lang:py%20pick.*%5C.memo%5Cb

__
(*) http://code.google.com/p/googleappengine/issues/detail?id=417

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Bill Janssen]

Josiah Carlson  wrote:

> On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing  
> wrote:
> > Daniel Stutzbach wrote:
> >
> >> If you have a working select(), it will tell you the sockets on which
> >> read() and write() won't block, so non-blocking reads and writes are not
> >> necessary.
> >
> > No, but there should be an interface that lets you say
> > "when something comes in on this fd, call this function
> > for me".
> >
> > In other words it should be a light wrapper around
> > select/poll/whatever that provides a callback interface.
> 
> A read callback, a write callback.  What about close, error, connect,
> and accept callbacks?
> 
> I hate to say it (not really), but that's pretty much the handle_*()
> methods of asyncore :/ .

What asyncore was missing was a timer API (a way to register functions
to be called periodically).  Then it would be pretty much like any other
event loop system.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib?

[Brett Cannon]

On Thu, Mar 5, 2009 at 12:52, Stefan Behnel  wrote:

> Benjamin Peterson wrote:
> > 2009/3/5 Guido van Rossum :
> >> On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel 
> wrote:
> >>> And, BTW, I wouldn't mind getting lxml into the stdlib either.
> >> No matter how beautiful and fast lxml is, it has one downside where it
> >> comes to installing it into the stdlib: it is based on large, complex
> >> 3rd party libraries, libxml2 and libxslt.
> >
> > And it depends on Cython, which is wonderful normally, but maybe
> > difficult to deal with in language evolution since we wouldn't have
> > direct control over the C sources.
>
> I see the point, although I think that this can be dealt with by
>
> a) using a specific, stable release version of Cython for a specific Python
> release, so that this Cython version can be bug fixed if required (it's
> implemented in Python, after all)
>

So including Cython source in the stdlib and then check in the generated C
code? I don't think that adding another build dependency for the stdlib,
especially for one already with several external dependencies itself, is a
good idea.


>
> or
>
> b) adding Cython to the stdlib and building with that


That's an entirely separate discussion (for which my initial answer is to
not consider it until it has stabilized to a 1.0 release).

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] running the tests...

[Nick Coghlan]

Chris Withers wrote:
> Nick Coghlan wrote:
>> My personal preferences:
>>
>> Thorough: ./python -m test.regrtest -uall
>> Typical: ./python -m test.regrtest
>> Specific: ./python -m test.regrtest test_mod1 test_mod2
> 
> This looks good, I assume this would work on Windows too?

Yep - and of course, you can even leave out the ./ in that case.

(Oh, if for some reason you're working on a Python 2.4 or earlier build,
the above won't work, since -m only started supporting running modules
inside packages in 2.5)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tracker cleanup - Roundup hacking report :)

[Daniel (ajax) Diniz]

Jesse Noller wrote:
> Slightly off topic Daniel, but if you see any multiprocessing bugs
> lurking out there, can you make me (jnoller) the assignee?

Sure!

FWIW, I've just submitted a patch[1] that will make working with
arbitrary issue sets much saner  and should have a 'mass-add user X as
nosy to selected issues' working locally before Saturday.

When these land on bugs.python.org, this kind of task will become so
easy that my job as tracker janitorveloper might become redundant :)

Cheers,
Daniel

[1]http://psf.upfronthosting.co.za/roundup/meta/issue246
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib?

[Stefan Behnel]

Benjamin Peterson wrote:
> 2009/3/5 Guido van Rossum :
>> On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel  wrote:
>>> And, BTW, I wouldn't mind getting lxml into the stdlib either.
>> No matter how beautiful and fast lxml is, it has one downside where it
>> comes to installing it into the stdlib: it is based on large, complex
>> 3rd party libraries, libxml2 and libxslt.
> 
> And it depends on Cython, which is wonderful normally, but maybe
> difficult to deal with in language evolution since we wouldn't have
> direct control over the C sources.

I see the point, although I think that this can be dealt with by

a) using a specific, stable release version of Cython for a specific Python
release, so that this Cython version can be bug fixed if required (it's
implemented in Python, after all)

or

b) adding Cython to the stdlib and building with that

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Josiah Carlson]

On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing  wrote:
> Daniel Stutzbach wrote:
>
>> If you have a working select(), it will tell you the sockets on which
>> read() and write() won't block, so non-blocking reads and writes are not
>> necessary.
>
> No, but there should be an interface that lets you say
> "when something comes in on this fd, call this function
> for me".
>
> In other words it should be a light wrapper around
> select/poll/whatever that provides a callback interface.

A read callback, a write callback.  What about close, error, connect,
and accept callbacks?

I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Daniel Stutzbach wrote:

If you have a working select(), it will tell you the sockets on which 
read() and write() won't block, so non-blocking reads and writes are not 
necessary.


No, but there should be an interface that lets you say
"when something comes in on this fd, call this function
for me".

In other words it should be a light wrapper around
select/poll/whatever that provides a callback interface.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Greg Ewing]

Neil Schemenauer wrote:


What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc) 


* a select() or epoll like interface that allows waiting on
  multiple channels


+1

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib?

[Stefan Behnel]

Guido van Rossum wrote:
> On Thu, Mar 5, 2009 at 11:22 AM, Stefan Behnel wrote:
>> I'm happy to see you jump onto this.
> 
> I'm not sure why you say that -- all I am doing is advising *against* 
> inclusion.

I understand that. What worth is a discussion where everyone just nods for
good? :)


>> Guido van Rossum wrote:
> There's *wy* too much stuff in the XML world to ever hope to have
> comprehensive support in the stdlib.

Definitely. But lxml was born because some Dutch guy thought that there was
way too little easy-to-master XML support in the overall Python world.

http://codespeak.net/lxml/intro.html

There is some space to look for a trade-off here.


>> It's good that (c)ElementTree is part of the stdlib, and it's also good
>> that there is a rather smooth upgrade path towards lxml.
> 
> And yet it worries me that lxml claims to be "mostly compatible" with
> ElementTree. What's keeping it from being completely (backwards)
> compatible?

The underlying tree model. An Element in lxml.etree knows it's parent,
which isn't the case in ET. That's the main difference. Most people call
that a feature in lxml, but it's fundamental and it does have the
implication that you can't keep the same Element object in more than one place.

Some other (minor) differences are described here:

http://codespeak.net/lxml/dev/compatibility.html


>> But lxml is by
>> itself becoming more and more a critical dependency of web related packages
>> and applications, simply because it provides everything in one tool.
> 
> That depends on how XML-centric your thinking is. Personally I *don't*
> like putting everything in XML, and so far I have been able to keep my
> code 99% XML-free.

That's totally fine. I used Python for years without ever feeling the need
to deploy any of the dbm databases in my projects. Nor curses, nor tk. And
lxml.objectify only supports pickle because one of the developers thought
it was a good idea to pickle trees. And yet all of these modules are part
of the stdlib, and I bet there are a whole lot of applications by now that
wouldn't work without them.


>> I would expect that even if lxml itself was in the stdlib, Linux
>> distributions would (want to) build it against their system libraries.
>> Static builds would only be required on MacOS-X and Windows.
> 
> And that in itself is one of the main arguments against inclusion in
> the stdlib, since it adds a whole new level of complexity to the
> compatibility matrix. E.g. assume that some newer version of libxml2
> has a new feature.

That happens. So far, I have managed to keep lxml backwards compatible over
more than three years of libxml2 releases. However:

> You can wrap that feature with an API in lxml, but
> now you require that newer libxml2 version as a dependency. Since the
> distros don't support that they either are prevented from providing
> the corresponding newer version of Python or you will have to make the
> lxml code conditional on the presence or absence of that API. The
> latter is preferable, but now it means that Python users can't rely on
> that API being present even if they have the right version of Python.
> It's a mess. Requiring a 3rd party download makes this cleaner,
> because you decouple the llibxml2/lxml versioning from the Python
> version.

A good example is actually (once again) parsing broken HTML. libxml2
handles this a lot better since 2.6.21, so if you use 2.6.20, you will
simply not get the same results as with a later version.

I do see the point you are making here. Even if lxml gets mature and
static, that doesn't necessarily apply to the external libraries it uses.
However, I should note that exactly the same argument also applies to
sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3 being
a fairly recent addition.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Benjamin Peterson]

2009/3/5 Guido van Rossum :
> On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel  wrote:
>> And, BTW, I wouldn't mind getting lxml into the stdlib either.
>
> No matter how beautiful and fast lxml is, it has one downside where it
> comes to installing it into the stdlib: it is based on large, complex
> 3rd party libraries, libxml2 and libxslt.

And it depends on Cython, which is wonderful normally, but maybe
difficult to deal with in language evolution since we wouldn't have
direct control over the C sources.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Pickler/Unpickler API clarification

[Collin Winter]

I'm working on some performance patches for cPickle, and one of the
bigger wins so far has been replacing the Pickler's memo dict with a
custom hashtable (and hence removing memo's getters and setters). In
looking over this, Jeffrey Yasskin commented that this would break
anyone who was accessing the memo attribute.

I've found a few examples of code using the memo attribute ([1], [2],
[3]), and there are probably more out there, but the memo attribute
doesn't look like part of the API to me. It's only documented in
http://docs.python.org/library/pickle.html as "you used to need this
before Python 2.3, but don't anymore". However: I don't believe you
should ever need this attribute.

The usages of memo I've seen break down into two camps: clearing the
memo, and wanting to explicitly populate the memo with predefined
values. Clearing the memo is recommended as part of reusing Pickler
objects, but I can't fathom when you would want to reuse a Pickler
*without* clearing the memo. Reusing the Pickler without clearing the
memo will produce pickles that are, as best I can see, invalid -- at
least, pickletools.dis() rejects this, which is the closest thing we
have to a validator. Explicitly setting memo values has the same
problem: an easy, very brittle way to produce invalid data.

So my questions are these:
1) Should Pickler/Unpickler objects automatically clear their memos
when dumping/loading?
2) Is memo an intentionally exposed, supported part of the
Pickler/Unpickler API, despite the lack of documentation and tests?

Thanks,
Collin

[1] - 
http://google.com/codesearch/p?hl=en#Qx8E-7HUBTk/trunk/google/appengine/api/memcache/__init__.py&q=lang:py%20%5C.memo
[2] - 
http://google.com/codesearch/p?hl=en#M-DDI-lCOgE/lib/python2.4/site-packages/cvs2svn_lib/primed_pickle.py&q=lang:py%20%5C.memo
[3] - 
http://google.com/codesearch/p?hl=en#l_w_cA4dKMY/AtlasAnalysis/2.0.3-LST-1/PhysicsAnalysis/PyAnalysis/PyAnalysisUtils/python/root_pickle.py&q=lang:py%20pick.*%5C.memo%5Cb
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and Queue.full()

[Jesse Noller]

On Thu, Mar 5, 2009 at 2:27 PM, Steve Holden  wrote:
> Jesse Noller wrote:
> [...]I'll take it from anyone.
>>
> And we can *quote* you on that?
>
> regards
>  Steve

As long as it's not on a t-shirt, I should be OK.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Daniel Stutzbach]

On Thu, Mar 5, 2009 at 1:30 PM, Neil Schemenauer  wrote:

> What I would like to see is a module that provides a low-level API
> for doing cross-platform asynchronous IO.  The two necessary parts
> are:
>
>* a wrapper that allows non-blocking reads and writes on
>  channels (sockets, file descriptors, serial ports, etc)
>
>* a select() or epoll like interface that allows waiting on
>  multiple channels
>

Two thoughts:

If you have a working select(), it will tell you the sockets on which read()
and write() won't block, so non-blocking reads and writes are not necessary.

On Windows, sockets, pipes, and files are each completely distinct types
with their own functions and unifying them under one select()-like interface
requires faking it using threads behind-the-scenes AFAIK.  Personally, I'd
be happy with continuing to only support socket objects on Windows.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib? (was: Integrate BeautifulSoup into stdlib?)

[Guido van Rossum]

On Thu, Mar 5, 2009 at 11:22 AM, Stefan Behnel  wrote:
> I'm happy to see you jump onto this.

I'm not sure why you say that -- all I am doing is advising *against* inclusion.

> Guido van Rossum wrote:
>> No matter how beautiful and fast lxml is, it has one downside where it
>> comes to installing it into the stdlib: it is based on large, complex
>> 3rd party libraries, libxml2 and libxslt.
>
> I actually had a recent discussion with other lxml developers and we were
> fast to agree that that would be the main problem. lxml itself is only some
> 18K lines of Cython code (which translates into 180K lines of C code) and
> less than 7K lines of Python code, but libxml2 and libxslt add up to about
> 230K lines of C code just by themselves. That is definitely far from
> trivial and it's hard to guarantee that bugs in these libraries will never
> lead to security holes in a Python release, for example.
>
> Still, it does provide an awful lot of things that the stdlib currently
> fails to deliver in one way or another, some even completely. XPath, XSLT,
> XML validation and (above all) real-world HTML parsing come to mind. I
> definitely stopped counting the posts on c.l.py about HTMLParser not being
> able to parse a specific web page.

There's *wy* too much stuff in the XML world to ever hope to have
comprehensive support in the stdlib. Heck, XmlPlus hasn't even been
incorporated into the stdlib.

> It's good that (c)ElementTree is part of the stdlib, and it's also good
> that there is a rather smooth upgrade path towards lxml.

And yet it worries me that lxml claims to be "mostly compatible" with
ElementTree. What's keeping it from being completely (backwards)
compatible?

> But lxml is by
> itself becoming more and more a critical dependency of web related packages
> and applications, simply because it provides everything in one tool.

That depends on how XML-centric your thinking is. Personally I *don't*
like putting everything in XML, and so far I have been able to keep my
code 99% XML-free.

> And
> even if I wasn't the author of lxml, I would have a hard time feeling happy
> if a real-world HTML parser was added to the stdlib that provides a totally
> different interface than the best (and fastest) XML library that the stdlib
> currently has.

That sounds like a completely different argument and one you should
have with the proponents of inclusion of that other parser. I can only
assume you're talking about html5lib or BeautifulSoup. I have no
knowledge of any of these, and prefer to stay out of that discussion.

>> Instead, let's hope Linux distros pick it up (and if anyone knows how
>> to encourage that, let us know).
>
> At least all Debian based distros (such as Ubuntu) have it available. Not
> the latest, greatest version, but that will come. That said, it's never
> been a real problem to EasyInstall lxml directly from PyPI onto any decent
> Linux distribution. MacOS-X is a far more tricky target here, not to say a
> word about Windows (C-compiler? anyone?).
>
> I would expect that even if lxml itself was in the stdlib, Linux
> distributions would (want to) build it against their system libraries.
> Static builds would only be required on MacOS-X and Windows.

And that in itself is one of the main arguments against inclusion in
the stdlib, since it adds a whole new level of complexity to the
compatibility matrix. E.g. assume that some newer version of libxml2
has a new feature. You can wrap that feature with an API in lxml, but
now you require that newer libxml2 version as a dependency. Since the
distros don't support that they either are prevented from providing
the corresponding newer version of Python or you will have to make the
lxml code conditional on the presence or absence of that API. The
latter is preferable, but now it means that Python users can't rely on
that API being present even if they have the right version of Python.
It's a mess. Requiring a 3rd party download makes this cleaner,
because you decouple the llibxml2/lxml versioning from the Python
version.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[Neil Schemenauer]

Chris McDonough  wrote:
> As far as I can tell, asyncore/asynchat is all "undocumented
> internals".  Any use of asyncore in anger will use internals;
> there never was any well-understood API to these modules.

What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc) 

* a select() or epoll like interface that allows waiting on
  multiple channels

The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.

  Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and Queue.full()

[Steve Holden]

Jesse Noller wrote:
[...]I'll take it from anyone.
> 
And we can *quote* you on that?

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate lxml into the stdlib? (was: Integrate BeautifulSoup into stdlib?)

[Stefan Behnel]

Hi Guido,

I'm happy to see you jump onto this.

Guido van Rossum wrote:
> No matter how beautiful and fast lxml is, it has one downside where it
> comes to installing it into the stdlib: it is based on large, complex
> 3rd party libraries, libxml2 and libxslt.

I actually had a recent discussion with other lxml developers and we were
fast to agree that that would be the main problem. lxml itself is only some
18K lines of Cython code (which translates into 180K lines of C code) and
less than 7K lines of Python code, but libxml2 and libxslt add up to about
230K lines of C code just by themselves. That is definitely far from
trivial and it's hard to guarantee that bugs in these libraries will never
lead to security holes in a Python release, for example.

Still, it does provide an awful lot of things that the stdlib currently
fails to deliver in one way or another, some even completely. XPath, XSLT,
XML validation and (above all) real-world HTML parsing come to mind. I
definitely stopped counting the posts on c.l.py about HTMLParser not being
able to parse a specific web page.

It's good that (c)ElementTree is part of the stdlib, and it's also good
that there is a rather smooth upgrade path towards lxml. But lxml is by
itself becoming more and more a critical dependency of web related packages
and applications, simply because it provides everything in one tool. And
even if I wasn't the author of lxml, I would have a hard time feeling happy
if a real-world HTML parser was added to the stdlib that provides a totally
different interface than the best (and fastest) XML library that the stdlib
currently has.


> Instead, let's hope Linux distros pick it up (and if anyone knows how
> to encourage that, let us know).

At least all Debian based distros (such as Ubuntu) have it available. Not
the latest, greatest version, but that will come. That said, it's never
been a real problem to EasyInstall lxml directly from PyPI onto any decent
Linux distribution. MacOS-X is a far more tricky target here, not to say a
word about Windows (C-compiler? anyone?).

I would expect that even if lxml itself was in the stdlib, Linux
distributions would (want to) build it against their system libraries.
Static builds would only be required on MacOS-X and Windows.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 372 -- Adding an ordered directory to collections ready for pronouncement

[Forest]

On Wed, Wed, 4 Mar 2009 13:52:59 -0800, Guido van Rossum wrote:
> On Wed, Mar 4, 2009 at 1:44 PM, Greg Ewing 
> wrote:
>> rdmur...@bitdance.com wrote:
>>
>>> I actually like StableDict best. ?When I hear that I think, "ah, the
>>> key order is stable in the face of insertions, unlike a regular dict".
>>
>> But it still doesn't convey what the ordering actually *is*.
>
> Please, stick with OrderedDict. That's the name used historically by
> most people who independently reinvented this functionality.

It's also what I typed into google and PYPI when I went looking for this
functionality.

+1 for odereddict or odict or OrderedDict.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Barry Warsaw]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 5, 2009, at 12:32 PM, Guido van Rossum wrote:


Instead, let's hope Linux distros pick it up (and if anyone knows how
to encourage that, let us know).



Gentoo: emerge lxml
Ubuntu (and probably Debian): apt-get install python-lxml

Guido, do you know where your time machine keys are? :)

Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbAaRHEjvBPtnXfVAQK3nQP/Uz6CF7zxIbTJWHyGyPBr1+pUTESzryNs
SKnBwcyIjYw/+7whtdfp31jbgsv+FcZ9YmMx7haUzPS/lKaRClvfUlirXepDCQt/
Z44nxvjEbbpQPmvlmf9SAIgvk7AumWcigXth2cvMJedHz0fVA9jXA1f/bnGxdTA6
/DUrqxruwZo=
=R5FW
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Guido van Rossum]

On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel  wrote:
> Ivan Krstić wrote:
>> On Mar 4, 2009, at 12:32 PM, James Y Knight wrote:
>>> I think html5lib would be a better candidate for an imrpoved HTML
>>> parser in the stdlib than BeautifulSoup.
>>
>> While we're talking about alternatives, Ian Bicking appears to swear by
>> lxml:
>>
>> 
>
> I second that. ;)
>
> And, BTW, I wouldn't mind getting lxml into the stdlib either.

No matter how beautiful and fast lxml is, it has one downside where it
comes to installing it into the stdlib: it is based on large, complex
3rd party libraries, libxml2 and libxslt.

Based on the sad example of BerkeleyDB, which was initially welcomed
into the stdlib but more recently booted out for reasons having to do
with the release cycle of the external dependency and other issues
typical for large external dependencies, I think we should be very
careful with including it in the standard library.

Instead, let's hope Linux distros pick it up (and if anyone knows how
to encourage that, let us know).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and Queue.full()

[Jesse Noller]

On Thu, Mar 5, 2009 at 1:43 AM, Tennessee Leeuwenburg
 wrote:
> Hi Jesse,
>
> I'm not sure what the most appropriate thing to do is. I could:
>   (a) leave any multiprocessing changes to you,
>   (b) alter the functioning of the method queue_empty inside
> test_multiprocessing to test for emptiness in a different manner
>
> I'm happy to go ahead and try my hand at making the appropriate changes but
> I don't want to tread in areas of the code that other people have ownership
> of.
>
> It appears as though the only place in multiprocessing which uses the
> deprecated call is in the test_multiprocessing file.
>
> I also found a call to Queue.empty in wsgui.py. I don't see any authorship
> tags at the top of this file but the last commiter was Andrew Kuchling. Do I
> need to contact him regarding this or is it appropriate for me to make the
> appropriate modifications without consulting him?
>
> Apologies to anyone if I appear to be overly tentative OR overly pushy -- I
> am aware that some people take a great deal of personal ownership of their
> works, while others are impatient with a soft approach.
>
> Regards,
> -Tennessee

I'm an equal opportunity patch employer: I'll take it from anyone.

That being said, I would open a new issue on the tracker
(bugs.python.org) outlining the issue (removing these/deprecating
them) and add me to the +noisy. There you can upload patches as you
create them, and we can all coordinate work and discussion there.

For the multiprocessing support - the first step is to identify why
it's being used and remove it (I'm head-deep in pycon talk prep, so I
haven't been able to look) and then replace the code. Second, I need
to make sure the multiprocessing.queue API does not have it's own APIs
for these (mp.queue is a clone of queue.queue) and deprecate those.

-jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tracker cleanup - Roundup hacking report :)

[jnoller]

On Mar 5, 2009 12:17pm, "Daniel (ajax) Diniz"  wrote:

Hi,


Here's a progress report on the "let's make the tracker a bit better"  
tasks.


...snip

Slightly off topic Daniel, but if you see any multiprocessing bugs lurking  
out there, can you make me (jnoller) the assignee?


jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Tracker cleanup - Roundup hacking report :)

[Daniel (ajax) Diniz]

Hi,
Here's a progress report on the "let's make the tracker a bit better" tasks.

Note: if you make use of saved queries, I recommend reading the
'anyone can remove any queries' issue:
http://psf.upfronthosting.co.za/roundup/meta/issue244

Feedback on meta-tracker open issues, as well as new RFEs and replies
to tracker-discuss threads should make the tracker meeting your needs
a bit more likely :)

Thanks Martin, Stefan, Jeroen, Ezio, Brett, Georg and others that
helped along the way.

Real tracker cleanup will be back soon ;)

Best regards,
Daniel

Open WIP tickets:
add # of comments to default issue list
Track the counts of messages and nosy users on a given issue.
http://psf.upfronthosting.co.za/roundup/meta/issue87
(screenshot: 
http://psf.upfronthosting.co.za/roundup/meta/file110/msg_nosy_counts.png
)

Add multiselects for searches
This allows one to search, e.g., for issues with "Version in
['2.5', '2.6', '2.7']" or "Stage in ['patch review', 'not selected']".
http://psf.upfronthosting.co.za/roundup/meta/issue243
(example: http://psf.upfronthosting.co.za/roundup/meta/file105/issue_search.html
)

Add 'Mail me' button to messages and issues
Similar to becoming nosy retroactively, by getting a past
message after the fact. This (supposedly) allows one to reply to any
message in an issue from email.
http://psf.upfronthosting.co.za/roundup/meta/issue245

Have issue ID search work from "search tracker" box
Passing an ID into the "search tracker" box opens the issue
with the matching ID.
http://psf.upfronthosting.co.za/roundup/meta/issue123

Add 'Stage' to results page
Adds 'Stage' as a column to the search results when 'Display'
is selected.
http://psf.upfronthosting.co.za/roundup/meta/issue242

"Search in All Issues" button
http://psf.upfronthosting.co.za/roundup/meta/issue224

Have CVS download of issue list put name rather than ID
http://issues.roundup-tracker.org/issue955070

Failed logins do not produce an error message
Add a 'Login as %(user) successful' green-banner on successful logins
http://psf.upfronthosting.co.za/roundup/meta/issue230

Tickets/ideas on sketching or local testing stage:

notation to filter by logged-in user
Allows 'reflexive' queries, like 'issues I've created',
'issues I follow', 'issues assigned to me'. Pending Query security
resolution.
http://issues.roundup-tracker.org/issue1525113

Shortcuts for selecting self in user lists: "Add me" for nosy and
"Claim" for assignee.

Provide prev/next/show list links when going through a list of bugs
http://psf.upfronthosting.co.za/roundup/meta/issue4

Ability to specify non-matching criteria in filters/searches
http://issues.roundup-tracker.org/issue678900

Email me my current changes without submitting (a.k.a. 'Save')

Allow searching for ranges for Number attributes
http://issues.roundup-tracker.org/issue1182919

input fields should have HTML id attributes
http://issues.roundup-tracker.org/issue1513369

Full text "AND" search has message scope, not issue scope
http://issues.roundup-tracker.org/issue1155657

Set attributes in the body
http://psf.upfronthosting.co.za/roundup/meta/issue198

Closed tickets:
Edit Queries screen should include a link to the search page
http://psf.upfronthosting.co.za/roundup/meta/issue202
Denote which dependencies issues have been closed
http://psf.upfronthosting.co.za/roundup/meta/issue207
Fix 'not closed' search
http://psf.upfronthosting.co.za/roundup/meta/issue240
Add 'Stage' to search page
http://psf.upfronthosting.co.za/roundup/meta/issue239
Add creator/assignee to search view
http://psf.upfronthosting.co.za/roundup/meta/issue180
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How do I get commit access?

[Brett Cannon]

On Thu, Mar 5, 2009 at 01:08, Chris Withers  wrote:

> Guido van Rossum wrote:
>
>> I'd like to jump in and vouch for Chris. I've known him for many years
>> and while we haven't worked closely I expect he'd be a valuable
>> contributor. So +1 from me for giving Chris commit privileges for core
>> Python.
>>
>
> Thanks :-) I can't promise how *much* time I'll be able to give, but when I
> do have itches, I'll certainly be scratching them...
>
>  (Chris, I assume you'll go through an apprentice phase where
>> you'll be letting another committer review your changes before you
>> commit them yourself.
>>
>
> How does Python work w.r.t. to dev? I'm used to the branch/merge pattern:
>
> - create branch, say chris.withers-issue4308
> - do work on that branch
> - request code review
> - merge branch to appropriate release branches and trunk
> - delete branch
>
> If this isn't the pattern Python uses, why isn't it? ;-)
>

Because Python started out on CVS and then moved to svn and just never
picked up the branching religion. But there is nothing wrong with creating a
branch in the sandbox and doing work there; just isn't common practice for
anything that isn't going to be a long term project.


>
>
>  Rietveld at codereview.appspot.com should be
>> helpful for getting your code reviewed. (Just use upload.py instead of
>> the Create Issue form. :-)
>>
>
> OK, although I'd prefer the branch/merge pattern, less toolage required...


Right, but that still doesn't preclude the usefulness of being able to leave
comments in Rietveld for a reviewer.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and Queue.full()

[Brett Cannon]

On Wed, Mar 4, 2009 at 17:30, Raymond Hettinger  wrote:

> Just noticed that the empty() and full() methods were still there.
>>> IIRC, we agreed to take them out (but leaving qsize() exposed).
>>> The docs entries and test cases were taken out, but the actual
>>> methods were accidentally left in.
>>>
>>
>> If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.
>>
>
> I recommend adding a warning to 3.0.2 and removing in 3.1.
> Waiting for more 3.x uptake doesn't serve our users well.
> IIRC, that was the rationale for cmp() removal in 3.0.1.


Right, but that's because 3.0.1 was released within four months of 3.0.0.
Who knows when 3.0.2 will be released. Having some users suddenly get a new
deprecation for some code just because they upgraded their interpreter
doesn't sound very backwards-compatible to me. This is true even for a
PendingDeprecationWarning.

-1 on tossing a warning into 3.0.2 and going +1 with the 3.1 deprecation.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] running the tests...

[Chris Withers]

s...@pobox.com wrote:

>> My personal preferences:
>> 
>> Thorough: ./python -m test.regrtest -uall

>> Typical: ./python -m test.regrtest
>> Specific: ./python -m test.regrtest test_mod1 test_mod2

Chris> This looks good, I assume this would work on Windows too?

I believe so, but you should still get a real OS.


I do use plenty of real operating systems, but one of python's biggest 
strengths for me is that it is totally cross platform...


...that and the soundcard in my desktop machine only has windows drivers :-(

*cough*

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] draft 3.1 release schedule

[Guilherme Polo]

On Thu, Mar 5, 2009 at 10:53 AM, Brad Miller  wrote:
>
>
> On Mon, Mar 2, 2009 at 4:22 PM, "Martin v. Löwis" 
> wrote:
>>
>> > Would whoever is responsible for IDLE please take a look at the patches
>> > I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively].
>> > These change the behavior of IDLE so that IDLESTARTUP or PYTHONSTARTUP
>> > files are executed with each restart. This allows loading frequently
>> > used packages, personal utilities, etc. automatically at each restart. I
>> > consider this a very important problem in IDLE, especially when using it
>> > to teach.
>>
>> Just to put this into perspective: I personally don't see that as a very
>> important problem. I didn't know IDLESTARTUP existed, and I use
>> PYTHONSTARTUP only for the command line (to setup readline and history).
>> I think there are many more open issues that are *way* more important.
>
> Martin,
>   No disrespect intended but I don't see how this puts things into
> perspective.  I'm writing to you from the annual computer science education
> conference (SIGCSE)  where Python is clearly gaining ground as an important
> language for teaching computer science.
> It seems logical to me that the committers are high powered Python users who
> don't think much about Python being used in education.  I'm just as
> frustrated as Mitchell about a patch for displaying ranges and
> dict_keys/values objects in a more user friendly way.  I submitted this
> patch during the 3.0 alpha phase and it is still sitting around.  For me
> this is a serious problem, but I can understand how it seems pretty minor to
> others, who are not teaching new programmers.
> So what is the solution?  The obvious solution is for one of us, that is
> someone who uses Python as an education tool, to become a committer.  This
> seems problematic to me.  Although I'm willing to be a committer, and I'm
> confident I have the development skills necessary to be a committer I don't
> have the time to develop the resume of patches needed to earn that
> privilege.
> It would be nice if we could find some solution to this.

Or... IDLE could be taken out from Python. Tkinter is following the
same path too, sadly.
My hope is that by removing IDLE from Python it would bring new
developers that are not necessary python developers (by this I mean
developers of python itself). I changed IDLE quite a bit last year,
but I'm not sure if anyone cared enough to look at it (added tabs, ttk
support, themes, window relayout, and some other things), and I don't
think continuing with it in the stdlib is bringing any benefits.

I have commit access, and although I have been inactive for two or
three weeks (maybe a bit more) now, I have submitted plenty of fixes
for tkinter which are mostly reviewed by Martin, and only, Martin --
when he has time to review or when the fix hits some level of
"important enough" to be looked at. I could just commit these fixes,
but some people would hate me then because I didn't let anyone review,
so I don't really think adding more new committers will bring the
benefits you are expecting.

A different problem also present in both tkinter and IDLE is the lack of tests.

> Brad
>>
>>
>> This is not to say that the patch should not applied - I haven't even
>> looked at it. It's just a warning that, if no other committer feels this
>> is as important as you fell it is, it may not be committed reviewed and
>> committed before 3.1.
>>
>> Regards,
>> Martin
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/bmiller%40luther.edu
>
>
>
> --
> Brad Miller
> Assistant Professor, Computer Science
> Luther College
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>
>



-- 
-- Guilherme H. Polo Goncalves
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] running the tests...

[skip]

>> My personal preferences:
>> 
>> Thorough: ./python -m test.regrtest -uall
>> Typical: ./python -m test.regrtest
>> Specific: ./python -m test.regrtest test_mod1 test_mod2

Chris> This looks good, I assume this would work on Windows too?

I believe so, but you should still get a real OS.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] draft 3.1 release schedule

[Brad Miller]

On Mon, Mar 2, 2009 at 4:22 PM, "Martin v. Löwis" wrote:

> > Would whoever is responsible for IDLE please take a look at the patches
> > I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively].
> > These change the behavior of IDLE so that IDLESTARTUP or PYTHONSTARTUP
> > files are executed with each restart. This allows loading frequently
> > used packages, personal utilities, etc. automatically at each restart. I
> > consider this a very important problem in IDLE, especially when using it
> > to teach.
>
> Just to put this into perspective: I personally don't see that as a very
> important problem. I didn't know IDLESTARTUP existed, and I use
> PYTHONSTARTUP only for the command line (to setup readline and history).
> I think there are many more open issues that are *way* more important.


Martin,

  No disrespect intended but I don't see how this puts things into
perspective.  I'm writing to you from the annual computer science education
conference (SIGCSE)  where Python is clearly gaining ground as an important
language for teaching computer science.

It seems logical to me that the committers are high powered Python users who
don't think much about Python being used in education.  I'm just as
frustrated as Mitchell about a patch for displaying ranges and
dict_keys/values objects in a more user friendly way.  I submitted this
patch during the 3.0 alpha phase and it is still sitting around.  For me
this is a serious problem, but I can understand how it seems pretty minor to
others, who are not teaching new programmers.

So what is the solution?  The obvious solution is for one of us, that is
someone who uses Python as an education tool, to become a committer.  This
seems problematic to me.  Although I'm willing to be a committer, and I'm
confident I have the development skills necessary to be a committer I don't
have the time to develop the resume of patches needed to earn that
privilege.

It would be nice if we could find some solution to this.

Brad


>
> This is not to say that the patch should not applied - I haven't even
> looked at it. It's just a warning that, if no other committer feels this
> is as important as you fell it is, it may not be committed reviewed and
> committed before 3.1.
>
> Regards,
> Martin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/bmiller%40luther.edu
>



-- 
Brad Miller
Assistant Professor, Computer Science
Luther College
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] running the tests...

[Chris Withers]

Nick Coghlan wrote:

My personal preferences:

Thorough: ./python -m test.regrtest -uall
Typical: ./python -m test.regrtest
Specific: ./python -m test.regrtest test_mod1 test_mod2


This looks good, I assume this would work on Windows too?

cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] running the tests...

[Nick Coghlan]

Georg Brandl wrote:
> Chris Withers schrieb:
>> Hi All,
>>
>> I found the very brief snippet on test-running at:
>>
>> http://python.org/dev/faq/#how-to-test-a-patch
>>
>> so thought I'd ask here:
>>
>> - what's the canonical way to run "all the tests"?
> 
> Assuming UNIXy OSes: make test, or if you want to save a bit of time,
> make quicktest.
> 
>> - what's the canonical way to run the tests for just the package being 
>> patched? (I'm assuming it's a standard library package here...)
> 
> In 90% of all cases, the test suite is called like the module, so
> 
>./python Lib/test/regrtest.py test_foo
> 
> where foo is the module name should do it.  In the other 10%, you'll have
> to look around a bit for the tests.  But since patching should always
> include adding a test, it's necessary anyway ;)

My personal preferences:

Thorough: ./python -m test.regrtest -uall
Typical: ./python -m test.regrtest
Specific: ./python -m test.regrtest test_mod1 test_mod2

(enabling the relevant test resources via -uall or something more
specific is especially important when working on things like the
networking code or the audio support - many of the relevant tests are
skipped by default)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] patch commit policies (was [issue4308] repr of httplib.IncompleteRead is stupid)

[Nick Coghlan]

Chris Withers wrote:
> That aside, is it actually a python-wide policy to *forbid* patching
> older releases where the patch isn't security-related?
> 
> I can understand the "no more releases unless there are security
> problems", but what's the harm in applying a patch to an old version
> branch on the off chance that a security release might be made some time?

Precisely because if we're doing a security release, we want to be sure
that that the *only* change that gets included is the fix to the
security problem. If there are other changes hanging around on the
branch, then the release would need to go at least a single release
candidate cycle (which we don't want to need to do for security releases
on old branches).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Stefan Behnel]

Ivan Krstić wrote:
> On Mar 4, 2009, at 12:32 PM, James Y Knight wrote:
>> I think html5lib would be a better candidate for an imrpoved HTML
>> parser in the stdlib than BeautifulSoup.
> 
> While we're talking about alternatives, Ian Bicking appears to swear by
> lxml:
> 
> 

I second that. ;)

And, BTW, I wouldn't mind getting lxml into the stdlib either.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] running the tests...

[Georg Brandl]

Chris Withers schrieb:
> Hi All,
> 
> I found the very brief snippet on test-running at:
> 
> http://python.org/dev/faq/#how-to-test-a-patch
> 
> so thought I'd ask here:
> 
> - what's the canonical way to run "all the tests"?

Assuming UNIXy OSes: make test, or if you want to save a bit of time,
make quicktest.

> - what's the canonical way to run the tests for just the package being 
> patched? (I'm assuming it's a standard library package here...)

In 90% of all cases, the test suite is called like the module, so

   ./python Lib/test/regrtest.py test_foo

where foo is the module name should do it.  In the other 10%, you'll have
to look around a bit for the tests.  But since patching should always
include adding a test, it's necessary anyway ;)

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 4 Mar, 09:36 pm, dan...@stutzbachenterprises.com wrote:
Will any or all of you be at PyCon?  I'd be willing to put in the extra 
work

to turn your notes into a PEP.


I definitely will be.  I'll see you there!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 4 Mar, 08:28 pm, ba...@python.org wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 4, 2009, at 2:44 PM, gl...@divmod.com wrote:
Maintaining compatibility with the 2.6.x version of asyncore 
presupposes that *someone* has written some software against that 
version of asyncore and it might break if they installed an upgrade, 
though.


FWIW, I use smtpd.py and a few of the asyncore APIs  (.loop(), 
.socket_map.clear(), and .close_all()) in the Mailman 3 test  suite. 
That only works on Python 2.6 and I don't recall even a hiccup  when 
moving from 2.5 to 2.6.


Well, in that case, I withdraw even my disinterested suggestion :).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

[glyph]

On 4 Mar, 09:14 pm, krs...@solarsail.hcs.harvard.edu wrote:
I spent about a half hour sometime in the last month talking this 
through with Itamar, though not in great detail. I'd be interested in 
sitting down with both of you and trying to establish more precisely 
how much work is necessary to get something to actually happen here. I 
won't outright promise a PEP, but I'll promise at the very least to 
write down elaborate notes that someone could turn into a PEP 
relatively straightforwardly. Deal?


Absolutely.  I really appreciate the offer.  As the other gentlemen 
suggested, PyCon would be an ideal venue for doing this.  Are you going 
to be there?


I'll hopefully be seeing your talk this evening, but I suspect that 
would be a pretty bad venue to try to get this done ;-).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] running the tests...

[Chris Withers]

Hi All,

I found the very brief snippet on test-running at:

http://python.org/dev/faq/#how-to-test-a-patch

...so thought I'd ask here:

- what's the canonical way to run "all the tests"?

- what's the canonical way to run the tests for just the package being 
patched? (I'm assuming it's a standard library package here...)


cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] patch commit policies (was [issue4308] repr of httplib.IncompleteRead is stupid)

[Chris Withers]

Martin v. Löwis wrote:

Martin v. Löwis  added the comment:


So all Chris has to do to get this applied to 2.5 is craft an exploit based
on the current behavior, right? ;-)


Right :-) Of course, security patches should see a much more careful
review than regular bug fixes.


Well, it's funny you say that, since where I bumped into this, the bug 
was effectively DOS'ing a couple of mailservers as a result of 
mailinglogger sending out log entries of uncaught exceptions such as 
this and so emitting 100Mb emails whenever the foreign server chose not 
to deliver the whole chunk requested...


That aside, is it actually a python-wide policy to *forbid* patching 
older releases where the patch isn't security-related?


I can understand the "no more releases unless there are security 
problems", but what's the harm in applying a patch to an old version 
branch on the off chance that a security release might be made some time?


cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How do I get commit access?

[Chris Withers]

Guido van Rossum wrote:

I'd like to jump in and vouch for Chris. I've known him for many years
and while we haven't worked closely I expect he'd be a valuable
contributor. So +1 from me for giving Chris commit privileges for core
Python. 


Thanks :-) I can't promise how *much* time I'll be able to give, but 
when I do have itches, I'll certainly be scratching them...



(Chris, I assume you'll go through an apprentice phase where
you'll be letting another committer review your changes before you
commit them yourself. 


How does Python work w.r.t. to dev? I'm used to the branch/merge pattern:

- create branch, say chris.withers-issue4308
- do work on that branch
- request code review
- merge branch to appropriate release branches and trunk
- delete branch

If this isn't the pattern Python uses, why isn't it? ;-)


Rietveld at codereview.appspot.com should be
helpful for getting your code reviewed. (Just use upload.py instead of
the Create Issue form. :-)


OK, although I'd prefer the branch/merge pattern, less toolage required...

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What type of object mmap.read_byte should return on py3k?

[Josiah Carlson]

On Sun, Mar 1, 2009 at 10:45 AM, Hirokazu Yamamoto
 wrote:
> I uploaded the patch with choice (a)
> http://bugs.python.org/file13215/py3k_mmap_and_bytes.patch
> If (b) is suitable, I'll rewrite the patch.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com
>

Has anyone been using mmap in python 3k to know what is more
intuitive?  When I was using mmap in python 2.4, I never used the
read/write methods, I stuck with slicing, which was very convenient
with 2.4 non-unicode strings.  I don't really have an intuition on 3.x
bytes.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com