Re: [Python-Dev] [Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py
Phillip.eby wrote: > Author: phillip.eby > Date: Tue Apr 18 02:59:55 2006 > New Revision: 45510 > > Modified: >python/trunk/Lib/pkgutil.py >python/trunk/Lib/pydoc.py > Log: > Second phase of refactoring for runpy, pkgutil, pydoc, and setuptools > to share common PEP 302 support code, as described here: > > http://mail.python.org/pipermail/python-dev/2006-April/063724.html Shouldn't this new module be named "pkglib" to be in line with the naming scheme used for all the other utility modules, e.g. httplib, imaplib, poplib, etc. ? > pydoc now supports PEP 302 importers, by way of utility functions in > pkgutil, such as 'walk_packages()'. It will properly document > modules that are in zip files, and is backward compatible to Python > 2.3 (setuptools installs for Python <2.5 will bundle it so pydoc > doesn't break when used with eggs.) Are you saying that the installation of setuptools in Python 2.3 and 2.4 will then overwrite the standard pydoc included with those versions ? I think that's the wrong way to go if not made an explicit option in the installation process or a separate installation altogether. I bothered by the fact that installing setuptools actually changes the standard Python installation by either overriding stdlib modules or monkey-patching them at setuptools import time. > What has not changed is that pydoc command line options do not support > zip paths or other importer paths, and the webserver index does not > support sys.meta_path. Those are probably okay as limitations. > > Tasks remaining: write docs and Misc/NEWS for pkgutil/pydoc changes, > and update setuptools to use pkgutil wherever possible, then add it > to the stdlib. Add setuptools to the stdlib ? I'm still missing the PEP for this along with the needed discussion touching among other things, the change of the distutils standard "python setup.py install" to install an egg instead of a site package. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 18 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] possible fix for recursive __call__ segfault
Hi Brett,
On Mon, Apr 17, 2006 at 05:34:16PM -0700, Brett Cannon wrote:
> + if (meth == self) {
> + PyErr_SetString(PyExc_RuntimeError,
> + "recursive __call__ definition");
> + return NULL;
> + }
This is not the proper way, as it can be worked around with a pair of
objects whose __call__ point to each other. The solution is to use the
counter of Py_{Enter,Leave}RecursiveCall(), as was done for old-style
classes (see classobject.c).
By the way, this is a known problem: the example you show is
Lib/test/crashers/infinite_rec_3.py, and the four other
infinite_rec_*.py are all slightly more subtle ways to trigger a similar
infinite loop in C. They point to the SF bug report at
http://python.org/sf/1202533, where we discuss the problem in general.
Basically, someone should try to drop many
Py_{Enter,Leave}RecursiveCall() pairs in the source until all the
currently-known bugs go away, and then measure if this has a noticeable
performance impact.
A bientot,
Mr. 8 Of The 12 Files In That Directory
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Martin v. Löwis wrote: > It can't be that simple. Python's stdout should indeed be inherited > from cmd.exe, but that, in turn, should have obtained it from > buildbot. So even though cmd.exe closes its handle, Python's handle > should still be fine. If buildbot then closes the other end of the > pipe, Python should get ERROR_BROKEN_PIPE. The only deadlock I > can see here is when buildbot does *not* close the pipe, but stops > reading from it. In that case, Python's WriteFile would block. > > If that happens, it would be useful to attach with a debugger to > find out where Python got stuck. I doubt it has anything to do with this issue, but I just thought I'd mention something strange I've encountered on Windows XP Pro (SP2) at work. If Python terminates due to an uncaught exception, with stdout & stderr redirected externally (ie within the batch file that started Python), the files that were redirected to cannot be deleted/renamed until the system is rebooted. If a bare except is used to trap any such exceptions (and the traceback printed explicitly) so that Python terminates normally, there is no problem (ie the redirected files can be deleted/renamed etc). I've never reported this as a Python bug because I've considered the antivirus SW likely to be the culprit. I don't recall seeing this with Windows 2000, but much was changed in the transition from the Win2k SOE to the WinXP SOE. A wild shot at best... Andrew. -- - Andrew I MacIntyre "These thoughts are mine alone..." E-mail: [EMAIL PROTECTED] (pref) | Snail: PO Box 370 [EMAIL PROTECTED] (alt) |Belconnen ACT 2616 Web:http://www.andymac.org/ |Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
why include Construct?* the struct module is very nice, but very limited and non-pythonic as well * pure python (no platform/security issues) IMHO this is a drawback. More on this below. * lots of people need to parse and build binary data structures, it's not an esoteric library * license: public domain* quite a large user base for such a short time (proves the need of the community) Indeed, I wish I had known about this a year ago; it would have saved me a lot of work. Of course it probably didn't exist a year ago... :( * easy to use and extend (follows the componentization pattern) * declarative: you don't need to write executable code for most cases Well, declarative is less flexible. OTOH declarative is nice in the way it is more readable and allows more optimisations. why not:* the code is (very) young. stable and all, but less than a month on the loose. * new features may still be added / existing ones may be changed in a non-backwards-compatible manner so why am i saying this now, instead of waiting a few months for it to maturet? well, i wanted to get feedback. those of you who have seen/used the library, please tell me what you think:* is it suitable for a standard library? * what more features would you want?* any changes you think are necessary? This is a very nice library indeed. But the number one feature that I need in something like this would be to use C. That's because of my application specific requirements, where i have observed that reapeatedly using struct.pack/unpack and reading bytes from a stream represents a considerable CPU overhead, whereas the same thing in C would be ultra fast. IMHO, at least in theory Construct could have small but fast C extension to take care of the encoding and decoding, which is the critical path. Everything else, like the declaration part, can be python, as it is usually done once on application startup. If you agree to go down this path I might even be able to volunteer some of my time to help, but it's not my decision. Best regards. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 343: confusing context terminology
PEP 343 says:
This PEP proposes that the protocol used by the with statement be
known as the "context management protocol", and that objects that
implement that protocol be known as "context managers". The term
"context" then encompasses all objects with a __context__() method
that returns a context manager (this means that all context managers
are contexts, but not all contexts are context managers).
I read this paragraph as meaning:
context = 'thing with __context__()'
context manager = 'thing returned by __context__()'
So in a 'with' statement:
with A as B:
...
A is a context; the context manager is internal and unavailable.
The PEP uses this terminology, but the documentation for contextlib
seems to reverse things. The decorator is called '@contextmanager',
not '@context', and the text says things like:
Note that you can use [EMAIL PROTECTED] to define a context
manager's \method{__context__} method. This is usually more convenient
than creating another class just to serve as a context. For example:
or:
\begin{funcdesc}{nested}{ctx1\optional{, ctx2\optional{, ...}}}
Combine multiple context managers into a single nested context manager.
ref/ref3.tex uses the terms one way, then reverses them:
A \dfn{context manager} is an object that manages the entry
to, and exit from, a \dfn{context} surrounding a block of
code. Context managers are normally invoked using the
\keyword{with} statement (described in section~\ref{with}),
but can also be used by directly invoking their methods.
\begin{methoddesc}[context manager]{__context__}{self}
Invoked when the object is used as the context expression of a
\keyword{with} statement. The return value must implement
\method{__enter__()} and \method{__exit__()} methods. Simple
context managers that wish to directly implement
\method{__enter__()} and \method{__exit__()} should just
return \var{self}.
Context managers written in Python can also implement this
method using a generator function decorated with the
\function{contextlib.contextmanager} decorator, as this can be
simpler than writing individual \method{__enter__()} and
\method{__exit__()} methods when the state to be managed is
complex.
Personally, I find this terminology counterintuitive. The "What's
New" uses 'context manager = thing with __context()' because I read
the first paragraph quoted from PEP 343, got confused, decided it
couldn't possibly mean what it said, and fell back
on the intuition that a 'manager' is usually a big
long-lived thing while a 'context' is more likely to be a transient
thing that exists for the duration of a statement and is then
discarded.
So, my questions:
1) Am I misunderstanding the PEP? Is
'context = 'thing with __context__()' really the PEP's
meaning?
2) If the answer to 1) is 'yes', can we change the PEP?
My intuition is that a 'manager' keeps track of multiple
entities being managed; to have a single 'context' produce
multiple 'context managers' over time seems exactly
backwards.
I think I understand the logic behind the name -- that the
context manager *manages* the entry to and exit from the
block -- but think 'Manager' is the wrong term for this
because the object is really encapsulating a
procedure/algorithm, not managing a set of objects. The
closest GoF pattern name might be 'Strategy', which is
still a bit of a stretch.
3) If the answer to 2) is "buzz off" :), the documentation
(lib/contextlib, RefGuide, and What's New) needs to be fixed
to use the terminology consistent with PEP 343. I'll do
the "What's New", but someone else would have to update the
other two.
--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
Indeed, I wish I had known about this a year ago; it would have saved
me a lot of work. Of course it probably didn't exist a year ago... :(
well, yeah. many people need "parsing-abilities", but they resort to ad-hoc parsers using struct/some ad-hoc implementation of their own. there clearly is a need for a generic, strong, and extensible parsing/building mechanism.
Well, declarative is less flexible. OTOH declarative is nice in the way it is more readable and allows more optimisations.
i don't think "less flexible" is the term. it's certainly different, but if you need something specific, you can always subclass a construct on your own. other than that, being declarative means easy to read/write/maintain/debug/upgrage (to a newer version of the library).
IMHO, at least in theory Construct could have small but fast C
extension to take care of the encoding and decoding, which is the
critical path. Everything else, like the declaration part, can be
python, as it is usually done once on application startup.
well, i expected the encodings package to have a str.encode("bin") and str.decode("bin")... for some reason there's no such codec. it's a pity.
This is a very nice library indeed. But the number one feature that I
need in something like this would be to use C. That's because of my
application specific requirements, where i have observed that
reapeatedly using struct.pack/unpack and reading bytes from a stream
represents a considerable CPU overhead, whereas the same thing in C
would be ultra fast.well, you must have the notion of a "stream", i.e., go back and forth, be able to read/write bits/bytes at arbitrary locations, etc. i thought of moving the library to pyrex, and compiling it, but the number of critical parts is very small -- basically only the Repeater class could be improved by writing it in C.
i mean, most of the time is consumed at creating objects in the objects tree, etc. for example, the Struct class simply iterates over the nested construsts and parses each of the in that sequence. doing a pythonic iteration of a C-level iteration over a pythonic object is practically the same.
If you agree to go down this path I might even be able to volunteer some of my time to help, but it's not my decision.
well, mainly i'm looking for ideas. just moving it to c wouldnt be too helpful. some ideas i have:* making the stream work with bytes instead of bits, so that memory consumption would decrease 8-fold... but then parsing unaligned fields (either by size of position) is gonna be a headache
* unifying the context tree with the parsing/building tree, to create less objects on the fly (but it has some issues)* using lambda functions for meta expressions, instead of eval(string) -- perhaps it's faster, but lambda is getting deprecated by python3k :(
apart from that, i'm rely on inheritance in many places. if some classes are written in C and some in python, i'm not sure how it could work (can a C class inherit a pythonic one? would it be easy to extend?). and, that means users would have to compile the C sources, while now all they have to do is extract a zip file. and then i'd have to write makefiles, and maintain those also... it's getting dirty. i like the painless "unzip-and-use" installation.
so if you have ideas, i'd be happy to hear those.thanks,-tomerOn 4/18/06, Gustavo Carneiro <
[EMAIL PROTECTED]> wrote:
why include Construct?* the struct module is very nice, but very limited and non-pythonic as well
* pure python (no platform/security issues) IMHO this is a drawback. More on this below.
* lots of people need to parse and build binary data structures, it's not an esoteric library
* license: public domain* quite a large user base for such a short time (proves the need of the community) Indeed, I wish I had known about this a year ago; it would have saved me a lot of work. Of course it probably didn't exist a year ago... :(
* easy to use and extend (follows the componentization pattern)
* declarative: you don't need to write executable code for most cases
Well, declarative is less flexible. OTOH declarative is nice in the way it is more readable and allows more optimisations.
why not:* the code is (very) young. stable and all, but less than a month on the loose. * new features may still be added / existing ones may be changed in a non-backwards-compatible manner
so why am i saying this now, instead of waiting a few months for it to maturet?
well, i wanted to get feedback.
those of you who have seen/used the library, please tell me what you think:* is it suitable for a standard library?
* what more features would you want?* any changes you think are necessary? This is a very nice library indeed. But the number one feature that I need in something like this would be to use C. That's because of my application specific requirements, where i have observed that reapeatedly using
struct.pack/unpack and reading bytes from a stream represents a considerable CPU overhead, whereas
Re: [Python-Dev] [Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py
At 10:55 AM 4/18/2006 +0200, M.-A. Lemburg wrote: >Phillip.eby wrote: > > Author: phillip.eby > > Date: Tue Apr 18 02:59:55 2006 > > New Revision: 45510 > > > > Modified: > >python/trunk/Lib/pkgutil.py > >python/trunk/Lib/pydoc.py > > Log: > > Second phase of refactoring for runpy, pkgutil, pydoc, and setuptools > > to share common PEP 302 support code, as described here: > > > > http://mail.python.org/pipermail/python-dev/2006-April/063724.html > >Shouldn't this new module be named "pkglib" to be in line with >the naming scheme used for all the other utility modules, e.g. httplib, >imaplib, poplib, etc. ? It's not a new module; it was previously a module with only one function in it, introduced in Python 2.3. If it were a new module, I'd have inquired about a name for it first. > > pydoc now supports PEP 302 importers, by way of utility functions in > > pkgutil, such as 'walk_packages()'. It will properly document > > modules that are in zip files, and is backward compatible to Python > > 2.3 (setuptools installs for Python <2.5 will bundle it so pydoc > > doesn't break when used with eggs.) > >Are you saying that the installation of setuptools in Python 2.3 >and 2.4 will then overwrite the standard pydoc included with >those versions ? Yes. As far as I can tell, there were no API changes to pydoc during this time, so this is effectively a "hot fix". This hot-fixing doesn't apply to setuptools system packages built with --root or --single-version-externally-managed, however, so OS vendors who build packages that wrap setuptools will have to make an explicit decision whether to also apply any fixes. If they do not, an end-user can of course install setuptools in their local PYTHONPATH and the hotfix will take effect. >I bothered by the fact that installing setuptools actually changes >the standard Python installation by either overriding stdlib modules >or monkey-patching them at setuptools import time. Please feel free to propose alternative solutions that will still ensure that setuptools "just works" for end-users. Both this and the pydoc hotfix are "practicality beats purity" issues. >Add setuptools to the stdlib ? I'm still missing the PEP for this >along with the needed discussion touching among other things, >the change of the distutils standard "python setup.py install" >to install an egg instead of a site package. Setuptools in the stdlib simply means that people wanting to use it can import it. It does not affect programs that do not import it. It also means that "python -m easy_install" is available without having to first download ez_setup.py. As for discussion, Guido originally brought up the question here a few months ago, and it's been listed in PEP 356 for a while. I've also posted things related to the inclusion both here and in distutils-sig. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343: confusing context terminology
At 10:04 AM 4/18/2006 -0400, A.M. Kuchling wrote: >PEP 343 says: > > This PEP proposes that the protocol used by the with statement be > known as the "context management protocol", and that objects that > implement that protocol be known as "context managers". The term > "context" then encompasses all objects with a __context__() method > that returns a context manager (this means that all context managers > are contexts, but not all contexts are context managers). > >I read this paragraph as meaning: > > context = 'thing with __context__()' > context manager = 'thing returned by __context__()' > >So in a 'with' statement: > >with A as B: > ... > >A is a context; the context manager is internal and unavailable. I believe this is backwards. But then again, I would think that, since I wrote the documentation that takes the opposite tack. :) Partly, that's because I agree with your intuition about long-livedness of "manager", but mostly it's because the PEP is where the "contextmanager" decorator originated, and because it doesn't make any sense to call the method __context__() if it returns a context *manager*. :) > 1) Am I misunderstanding the PEP? Is >'context = 'thing with __context__()' really the PEP's >meaning? > > 2) If the answer to 1) is 'yes', can we change the PEP? > >My intuition is that a 'manager' keeps track of multiple >entities being managed; to have a single 'context' produce >multiple 'context managers' over time seems exactly >backwards. > >I think I understand the logic behind the name -- that the >context manager *manages* the entry to and exit from the >block -- but think 'Manager' is the wrong term for this >because the object is really encapsulating a >procedure/algorithm, not managing a set of objects. The >closest GoF pattern name might be 'Strategy', which is >still a bit of a stretch. I think we should correct the PEP. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Place for setuptools manuals and source for .exe files?
Now that setuptools is in the trunk, I need to also add its manuals and the source for its .exe files. These currently live only in the sandbox. First, the C source, used to create the 'gui.exe' and 'cli.exe' launchers that setuptools uses to create script wrappers on Windows. The source currently lives at sandbox/trunk/setuptools/launcher.c - should I create a PC/setuptools directory (analagous to PC/bdist_wininst) to put it in? Do I need to make it buildable with the MS toolchain? I currently build it with MinGW, as the code doesn't link with Python at all; it's strictly a standalone .exe, like w9xpopen. And its usage is similar to bdist_wininst's .exe header, in that it's used by setuptools as data during runtime, and the .exe is directly kept in revision control. So it doesn't need to be a part of the normal build process, but would only be rebuilt if there are any changes to the source. Second, the manuals. Setuptools currently has three: pkg_resources.txt -- the reference manual for the pkg_resources module setuptools.txt -- a developer's guide to using setuptools to package projects for distribution EasyInstall.txt -- a user's manual for the easy_install package installation tool. It's pretty clear that pkg_resources.txt should be converted to a standard library reference chapter, since that's essentially what it is. The other two manuals, however, are roughly the setuptools versions of "distributing Python modules" and "installing Python modules", respectively. Should they be listed as separate manuals? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Updated: PEP 359: The make statement
I've updated PEP 359 with a bunch of the recent suggestions. The
patch is available at:
http://bugs.python.org/1472459
and I've pasted the full text below.
I've tried to be more explicit about the goals -- the make statement
is mostly syntactic sugar for::
class :
__metaclass__ =
so that you don't have to lie to your readers when you're not actually
creating a class. I've also added some new examples and expanded the
discussion of the old ones to give the statement some better
motivation. And I've expanded the Open Issues discussions to consider
a few alternate keywords and to indicate some of the difficulties in
allowing a ``__make_dict__`` attribute for customizing the dict in
which the block is executed.
PEP: 359
Title: The "make" Statement
Version: $Revision: 45366 $
Last-Modified: $Date: 2006-04-13 07:36:24 -0600 (Thu, 13 Apr 2006) $
Author: Steven Bethard <[EMAIL PROTECTED]>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 05-Apr-2006
Python-Version: 2.6
Post-History: 05-Apr-2006, 06-Apr-2006, 13-Apr-2006
Abstract
This PEP proposes a generalization of the class-declaration syntax,
the ``make`` statement. The proposed syntax and semantics parallel
the syntax for class definition, and so::
make :
is translated into the assignment::
= ("", , )
where is the dict created by executing .
This is mostly syntactic sugar for::
class :
__metaclass__ =
and is intended to help more clearly express the intent of the
statement when something other than a class is being created. Of
course, other syntax for such a statement is possible, but it is
hoped that by keeping a strong parallel to the class statement, an
understanding of how classes and metaclasses work will translate into
an understanding of how the make-statement works as well.
The PEP is based on a suggestion [1]_ from Michele Simionato on the
python-dev list.
Motivation
==
Class statements provide two nice facilities to Python:
(1) They execute a block of statements and provide the resulting
bindings as a dict to the metaclass.
(2) They encourage DRY (don't repeat yourself) by allowing the class
being created to know the name it is being assigned.
Thus in a simple class statement like::
class C(object):
x = 1
def foo(self):
return 'bar'
the metaclass (``type``) gets called with something like::
C = type('C', (object,), {'x':1, 'foo':})
The class statement is just syntactic sugar for the above assignment
statement, but clearly a very useful sort of syntactic sugar. It
avoids not only the repetition of ``C``, but also simplifies the
creation of the dict by allowing it to be expressed as a series of
statements.
Historically, type instances (a.k.a. class objects) have been the
only objects blessed with this sort of syntactic support. The make
statement aims to extend this support to other sorts of objects where
such syntax would also be useful.
Example: simple namespaces
--
Let's say I have some attributes in a module that I access like::
mod.thematic_roletype
mod.opinion_roletype
mod.text_format
mod.html_format
and since "Namespaces are one honking great idea", I'd like to be
able to access these attributes instead as::
mod.roletypes.thematic
mod.roletypes.opinion
mod.format.text
mod.format.html
I currently have two main options:
(1) Turn the module into a package, turn ``roletypes`` and
``format`` into submodules, and move the attributes to
the submodules.
(2) Create ``roletypes`` and ``format`` classes, and move the
attributes to the classes.
The former is a fair chunk of refactoring work, and produces two
tiny modules without much content. The latter keeps the attributes
local to the module, but creates classes when there is no intention
of ever creating instances of those classes.
In situations like this, it would be nice to simply be able to
declare a "namespace" to hold the few attributes. With the new make
statement, I could introduce my new namespaces with something like::
make namespace roletypes:
thematic = ...
opinion = ...
make namespace format:
text = ...
html = ...
and keep my attributes local to the module without making classes
that are never intended to be instantiated. One definition of
namespace that would make this work is::
class namespace(object):
def __init__(self, name, args, kwargs):
self.__dict__.update(kwargs)
Given this definition, at the end of the make-statements above,
``roletypes`` and ``format`` would be namespace instances.
Example: gui objects
In gui toolkits, objects like frames and panels are often associated
with attributes and functions. With the make-statement, code that
looks something like::
root = Tkinter.Tk()
frame = Tkinter.Frame(
Re: [Python-Dev] setuptools in the stdlib ([Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Phillip J. Eby wrote: > At 10:55 AM 4/18/2006 +0200, M.-A. Lemburg wrote: >> Phillip.eby wrote: >> > Author: phillip.eby >> > Date: Tue Apr 18 02:59:55 2006 >> > New Revision: 45510 >> > >> > Modified: >> >python/trunk/Lib/pkgutil.py >> >python/trunk/Lib/pydoc.py >> > Log: >> > Second phase of refactoring for runpy, pkgutil, pydoc, and setuptools >> > to share common PEP 302 support code, as described here: >> > >> > http://mail.python.org/pipermail/python-dev/2006-April/063724.html >> >> Shouldn't this new module be named "pkglib" to be in line with >> the naming scheme used for all the other utility modules, e.g. httplib, >> imaplib, poplib, etc. ? > > It's not a new module; it was previously a module with only one function > in it, introduced in Python 2.3. If it were a new module, I'd have > inquired about a name for it first. Didn't realize that. Too bad the time machine didn't work on this one :-/ >> > pydoc now supports PEP 302 importers, by way of utility functions in >> > pkgutil, such as 'walk_packages()'. It will properly document >> > modules that are in zip files, and is backward compatible to Python >> > 2.3 (setuptools installs for Python <2.5 will bundle it so pydoc >> > doesn't break when used with eggs.) >> >> Are you saying that the installation of setuptools in Python 2.3 >> and 2.4 will then overwrite the standard pydoc included with >> those versions ? > > Yes. As far as I can tell, there were no API changes to pydoc during > this time, so this is effectively a "hot fix". Why should a 3rd party extension be hot-fixing the standard Python distribution ? > This hot-fixing doesn't apply to setuptools system packages built with > --root or --single-version-externally-managed, however, so OS vendors > who build packages that wrap setuptools will have to make an explicit > decision whether to also apply any fixes. If they do not, an end-user > can of course install setuptools in their local PYTHONPATH and the > hotfix will take effect. What does setuptools have to do with pydoc ? Why should a user installing setuptools assume that some arbitrary stdlib modules get (effectively) replaced by installing setuptools ? If you want to provide a hot fix for Python 2.3 and 2.4, you should make this a separate install, so that users are aware that their Python distribution is about to get modified in ways that have nothing to do with setuptools. >> I bothered by the fact that installing setuptools actually changes >> the standard Python installation by either overriding stdlib modules >> or monkey-patching them at setuptools import time. > > Please feel free to propose alternative solutions that will still ensure > that setuptools "just works" for end-users. Both this and the pydoc > hotfix are "practicality beats purity" issues. Not really. I'd consider them design flaws. distutils is built to be extended without having to monkey-patch it, e.g. you can easily override commands with your own variants by supplying them via the cmdclass and distclass keyword arguments to setup(). By monkey patching distutils during import of setuptools, you effectively *change* distutils at run-time and not only for the application space that you implement in setuptools, but for all the rest of the application. If an application wants to use setuptools for e.g. plugin management, then standard distutils features will get replaced by setuptools implementations which are not compatible to the standard distutils commands, effectively making it impossible to access the original versions. Monkey patching is only a last resort in case nothing else works. In this case, it's not needed, since distutils provides the interfaces needed to extend its command classes via the setup() call. See e.g. mxSetup.py in the eGenix extensions for an example of how effective the distutils design can be put to use without having to introduce lots of unwanted side-effects. >> Add setuptools to the stdlib ? I'm still missing the PEP for this >> along with the needed discussion touching among other things, >> the change of the distutils standard "python setup.py install" >> to install an egg instead of a site package. > > Setuptools in the stdlib simply means that people wanting to use it can > import it. It does not affect programs that do not import it. It also > means that "python -m easy_install" is available without having to first > download ez_setup.py. Doesn't really seem like much of an argument for the addition... the above is true for any 3rd party module. > As for discussion, Guido originally brought up the question here a few > months ago, and it's been listed in PEP 356 for a while. I've also > posted things related to the inclusion both here and in distutils-sig. I know, but the discussions haven't really helped much in getting the setuptools design compatible with standard distutils. Unless that's being put in place, I'm -1 on the addition, due to the invasive nature of setuptools and its vari
Re: [Python-Dev] 2.5a1 Performance
Anthony Baxter wrote: > On Thursday 06 April 2006 04:10, Benji York wrote: >> On a related note: it might be nice to put a pystone run in the >> buildbot so it'd be easier to compare pystones across different >> releases, different architectures, and between particular changes >> to the code. (That's assuming that the machines are otherwise idle, >> though.) -- > -1. > > A bad benchmark (which pystone is) is much worse than no benchmark. I could contribute pybench to the Tools/ directory if that makes a difference: pybench -- The Python Benchmark Suite Extendable suite of of low-level benchmarks for measuring the performance of the Python implementation (interpreter, compiler or VM). WHAT IS IT ?: pybench is a collection of tests that provides a standardized way to measure the performance of Python implementations. It takes a very close look at different aspects of Python programs and let's you decide which factors are more important to you than others, rather than wrapping everything up in one number, like the other performance tests do (e.g. pystone which is included in the Python Standard Library). pybench has been used in the past by several Python developers to track down performance bottlenecks or to demonstrate the impact of optimizations and new features in Python. There's currently no documentation and no distutils support in pybench; that'll go into one of the next releases. For now, please read the source code. The command line interface for pybench is the file pybench.py. Run this script with option '--help' to get a listing of the possible options. Without options, pybench will simply execute the benchmark and then print out a report to stdout. Here's some sample output: """ Tests: per runper oper. overhead -- BuiltinFunctionCalls: 131.50 ms1.03 us0.50 ms BuiltinMethodLookup: 195.85 ms0.37 us1.00 ms CompareFloats: 126.00 ms0.28 us1.00 ms CompareFloatsIntegers: 201.05 ms0.45 us0.50 ms CompareIntegers: 192.05 ms0.21 us2.00 ms CompareInternedStrings: 117.65 ms0.24 us3.50 ms ... TryExcept: 289.75 ms0.19 us3.00 ms TryRaiseExcept: 179.05 ms 11.94 us1.00 ms TupleSlicing: 159.75 ms1.52 us0.50 ms UnicodeMappings: 171.85 ms9.55 us1.00 ms UnicodePredicates: 152.05 ms0.68 us4.00 ms UnicodeProperties: 203.00 ms1.01 us4.00 ms UnicodeSlicing: 190.10 ms1.09 us2.00 ms -- Average round time: 10965.00 ms """ This is the current version: http://www.lemburg.com/files/python/pybench-1.2.zip -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 18 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ([Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
At 07:15 PM 4/18/2006 +0200, M.-A. Lemburg wrote: >Why should a 3rd party extension be hot-fixing the standard >Python distribution ? Because setuptools installs things in zip files, and older versions of pydoc don't work for packages zip files. >If you want to provide a hot fix for Python 2.3 and 2.4, you >should make this a separate install, so that users are aware >that their Python distribution is about to get modified in ways >that have nothing to do with setuptools. Their Python distribution is not "modified" -- new modules are merely placed on sys.path ahead of the stdlib. (Which, I might add, is a perfectly normal process in Python -- nothing stops users from installing their own version of pydoc or any other module via PYTHONPATH.) Note also that uninstalling setuptools by removing the .egg file or directory will effectively remove the hot fix, since the modules live in the .egg, not in the stdlib. >If an application wants to use setuptools for e.g. plugin >management, then standard distutils features will get >replaced by setuptools implementations which are not compatible >to the standard distutils commands, effectively making it >impossible to access the original versions. Please do a little research before you spread FUD. The 'pkg_resources' module is used for runtime plugin management, and it does not monkeypatch anything. >Monkey patching is only a last resort in case nothing >else works. In this case, it's not needed, since distutils >provides the interfaces needed to extend its command classes >via the setup() call. The monkeypatching is there so that the easy_install command can build eggs for packages that use the distutils. It's also there so that other distutils extensions that monkeypatch distutils (and there are a few of them out there) will have a better chance of working with setuptools. I originally took a minimally-invasive approach to setuptools-distutils interaction, but it was simply not possible to provide a high-degree of backward and "sideward" compatibility without it. In fact, I seem to recall finding some behaviors in some versions of distutils that can't be modified without monkeypatching, although the details are escaping me at this moment. > > As for discussion, Guido originally brought up the question here a few > > months ago, and it's been listed in PEP 356 for a while. I've also > > posted things related to the inclusion both here and in distutils-sig. > >I know, but the discussions haven't really helped much in >getting the setuptools design compatible with standard >distutils. That's because the job was already done. :) The setuptools design bends over backwards to be compatible with Python 2.3 and 2.4 versions of distutils, not to mention py2exe, Pyrex, and other distutils extensions, along with the quirky uses of distutils that exist in dozens of distributed Python packages. However, I think you and I may perhaps have different definitions of "compatibility". Mine is that things "just work" and users don't have to do anything special. For that definition, setuptools is extremely compatible with the standard distutils. In many situations it's more compatible than the distutils themselves, in that more things "just work". ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5a1 Performance
On 4/18/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: > Anthony Baxter wrote: > > On Thursday 06 April 2006 04:10, Benji York wrote: > >> On a related note: it might be nice to put a pystone run in the > >> buildbot so it'd be easier to compare pystones across different > >> releases, different architectures, and between particular changes > >> to the code. (That's assuming that the machines are otherwise idle, > >> though.) -- > > -1. > > > > A bad benchmark (which pystone is) is much worse than no benchmark. > > I could contribute pybench to the Tools/ directory if that > makes a difference: I'd find that helpful. Jeremy > > pybench -- The Python Benchmark Suite > > Extendable suite of of low-level benchmarks for measuring > the performance of the Python implementation > (interpreter, compiler or VM). > > > > WHAT IS IT ?: > > pybench is a collection of tests that provides a standardized way > to measure the performance of Python implementations. It takes a > very close look at different aspects of Python programs and let's > you decide which factors are more important to you than others, > rather than wrapping everything up in one number, like the other > performance tests do (e.g. pystone which is included in the Python > Standard Library). > > pybench has been used in the past by several Python developers to > track down performance bottlenecks or to demonstrate the impact > of optimizations and new features in Python. > > There's currently no documentation and no distutils support in > pybench; that'll go into one of the next releases. For now, > please read the source code. The command line interface for pybench > is the file pybench.py. Run this script with option '--help' > to get a listing of the possible options. Without options, > pybench will simply execute the benchmark and then print out > a report to stdout. Here's some sample output: > """ > Tests: per runper oper. overhead > -- > BuiltinFunctionCalls: 131.50 ms1.03 us0.50 ms >BuiltinMethodLookup: 195.85 ms0.37 us1.00 ms > CompareFloats: 126.00 ms0.28 us1.00 ms > CompareFloatsIntegers: 201.05 ms0.45 us0.50 ms >CompareIntegers: 192.05 ms0.21 us2.00 ms > CompareInternedStrings: 117.65 ms0.24 us3.50 ms > ... > TryExcept: 289.75 ms0.19 us3.00 ms > TryRaiseExcept: 179.05 ms 11.94 us1.00 ms > TupleSlicing: 159.75 ms1.52 us0.50 ms >UnicodeMappings: 171.85 ms9.55 us1.00 ms > UnicodePredicates: 152.05 ms0.68 us4.00 ms > UnicodeProperties: 203.00 ms1.01 us4.00 ms > UnicodeSlicing: 190.10 ms1.09 us2.00 ms > -- > Average round time: 10965.00 ms > """ > > This is the current version: > > http://www.lemburg.com/files/python/pybench-1.2.zip > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Apr 18 2006) > >>> Python/Zope Consulting and Support ...http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ > > > ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5a1 Performance
M.-A. Lemburg wrote: >I could contribute pybench to the Tools/ directory if that >makes a difference: > > > +1 on adding pybench. And we already have parrotbench in the sandbox. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5a1 Performance
[M.-A. Lemburg] > I could contribute pybench to the Tools/ directory if that > makes a difference: +1. It's frequently used and nice work. Besides, then we could easily fiddle the tests to make Python look better ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] possible fix for recursive __call__ segfault
On 4/18/06, Armin Rigo <[EMAIL PROTECTED]> wrote:
> Hi Brett,
>
> On Mon, Apr 17, 2006 at 05:34:16PM -0700, Brett Cannon wrote:
> > + if (meth == self) {
> > + PyErr_SetString(PyExc_RuntimeError,
> > + "recursive __call__ definition");
> > + return NULL;
> > + }
>
> This is not the proper way, as it can be worked around with a pair of
> objects whose __call__ point to each other.
Yeah, I know. It was just a quick hack that at least helped me
identify where the problem was. I didn't really expect for it to
stick around.
> The solution is to use the
> counter of Py_{Enter,Leave}RecursiveCall(), as was done for old-style
> classes (see classobject.c).
>
OK. Makes sense.
> By the way, this is a known problem: the example you show is
> Lib/test/crashers/infinite_rec_3.py, and the four other
> infinite_rec_*.py are all slightly more subtle ways to trigger a similar
> infinite loop in C. They point to the SF bug report at
> http://python.org/sf/1202533, where we discuss the problem in general.
> Basically, someone should try to drop many
> Py_{Enter,Leave}RecursiveCall() pairs in the source until all the
> currently-known bugs go away, and then measure if this has a noticeable
> performance impact.
>
OK, good to know. I will have to fix this eventually for sandboxing
reasons for my dissertation so I will get to it eventually.
-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Updated: PEP 359: The make statement
Definitely an intriguing idea! I am +0 just because I don't know how
needed it is, but it is definitely cool.
As for your open issues, ditching __metaclass__ is fine if this goes
in, but I would keep 'class' around as simplified syntactic sugar for
the common case.
-Brett
On 4/18/06, Steven Bethard <[EMAIL PROTECTED]> wrote:
> I've updated PEP 359 with a bunch of the recent suggestions. The
> patch is available at:
> http://bugs.python.org/1472459
> and I've pasted the full text below.
>
> I've tried to be more explicit about the goals -- the make statement
> is mostly syntactic sugar for::
>
>class :
>__metaclass__ =
>
>
> so that you don't have to lie to your readers when you're not actually
> creating a class. I've also added some new examples and expanded the
> discussion of the old ones to give the statement some better
> motivation. And I've expanded the Open Issues discussions to consider
> a few alternate keywords and to indicate some of the difficulties in
> allowing a ``__make_dict__`` attribute for customizing the dict in
> which the block is executed.
>
>
>
> PEP: 359
> Title: The "make" Statement
> Version: $Revision: 45366 $
> Last-Modified: $Date: 2006-04-13 07:36:24 -0600 (Thu, 13 Apr 2006) $
> Author: Steven Bethard <[EMAIL PROTECTED]>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 05-Apr-2006
> Python-Version: 2.6
> Post-History: 05-Apr-2006, 06-Apr-2006, 13-Apr-2006
>
>
> Abstract
>
>
> This PEP proposes a generalization of the class-declaration syntax,
> the ``make`` statement. The proposed syntax and semantics parallel
> the syntax for class definition, and so::
>
>make :
>
>
> is translated into the assignment::
>
> = ("", , )
>
> where is the dict created by executing .
> This is mostly syntactic sugar for::
>
>class :
>__metaclass__ =
>
>
> and is intended to help more clearly express the intent of the
> statement when something other than a class is being created. Of
> course, other syntax for such a statement is possible, but it is
> hoped that by keeping a strong parallel to the class statement, an
> understanding of how classes and metaclasses work will translate into
> an understanding of how the make-statement works as well.
>
> The PEP is based on a suggestion [1]_ from Michele Simionato on the
> python-dev list.
>
>
> Motivation
> ==
>
> Class statements provide two nice facilities to Python:
>
> (1) They execute a block of statements and provide the resulting
> bindings as a dict to the metaclass.
>
> (2) They encourage DRY (don't repeat yourself) by allowing the class
> being created to know the name it is being assigned.
>
> Thus in a simple class statement like::
>
> class C(object):
> x = 1
> def foo(self):
> return 'bar'
>
> the metaclass (``type``) gets called with something like::
>
> C = type('C', (object,), {'x':1, 'foo':})
>
> The class statement is just syntactic sugar for the above assignment
> statement, but clearly a very useful sort of syntactic sugar. It
> avoids not only the repetition of ``C``, but also simplifies the
> creation of the dict by allowing it to be expressed as a series of
> statements.
>
> Historically, type instances (a.k.a. class objects) have been the
> only objects blessed with this sort of syntactic support. The make
> statement aims to extend this support to other sorts of objects where
> such syntax would also be useful.
>
>
> Example: simple namespaces
> --
>
> Let's say I have some attributes in a module that I access like::
>
> mod.thematic_roletype
> mod.opinion_roletype
>
> mod.text_format
> mod.html_format
>
> and since "Namespaces are one honking great idea", I'd like to be
> able to access these attributes instead as::
>
> mod.roletypes.thematic
> mod.roletypes.opinion
>
> mod.format.text
> mod.format.html
>
> I currently have two main options:
>
> (1) Turn the module into a package, turn ``roletypes`` and
> ``format`` into submodules, and move the attributes to
> the submodules.
>
> (2) Create ``roletypes`` and ``format`` classes, and move the
> attributes to the classes.
>
> The former is a fair chunk of refactoring work, and produces two
> tiny modules without much content. The latter keeps the attributes
> local to the module, but creates classes when there is no intention
> of ever creating instances of those classes.
>
> In situations like this, it would be nice to simply be able to
> declare a "namespace" to hold the few attributes. With the new make
> statement, I could introduce my new namespaces with something like::
>
> make namespace roletypes:
> thematic = ...
> opinion = ...
>
> make namespace format:
> text = ...
> html = ...
>
> and keep my attributes local to the module without making classes
> that are never
Re: [Python-Dev] PEP 343: confusing context terminology
On 4/18/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > I think we should correct the PEP. Yes please, go ahead. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ([Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Phillip J. Eby wrote: > At 07:15 PM 4/18/2006 +0200, M.-A. Lemburg wrote: >> Why should a 3rd party extension be hot-fixing the standard >> Python distribution ? > > Because setuptools installs things in zip files, and older versions of > pydoc don't work for packages zip files. That doesn't answer my question. >> If you want to provide a hot fix for Python 2.3 and 2.4, you >> should make this a separate install, so that users are aware >> that their Python distribution is about to get modified in ways >> that have nothing to do with setuptools. > > Their Python distribution is not "modified" -- new modules are merely > placed on sys.path ahead of the stdlib. (Which, I might add, is a > perfectly normal process in Python -- nothing stops users from installing > their own version of pydoc or any other module via PYTHONPATH.) > > Note also that uninstalling setuptools by removing the .egg file or > directory will effectively remove the hot fix, since the modules live in > the .egg, not in the stdlib. Whether you place a module with the same name in front of the stdlib path in PYTHONPATH (e.g. copy it into site-packages) or replace the file in the Python installation is really the same thing to the user. Third-party extension *should not do this* ! It's OK to have private modified copies of a module inside a package or used inside an application, but "python setup.py install" should never (effectively) replace a Python stdlib module with some modified copy without explicit user interaction. >> If an application wants to use setuptools for e.g. plugin >> management, then standard distutils features will get >> replaced by setuptools implementations which are not compatible >> to the standard distutils commands, effectively making it >> impossible to access the original versions. > > Please do a little research before you spread FUD. The 'pkg_resources' > module is used for runtime plugin management, and it does not monkeypatch > anything. I'm talking about the setuptools package which does apply monkey patching and is needed to manage the download and installation of plugin eggs, AFAIK. >> Monkey patching is only a last resort in case nothing >> else works. In this case, it's not needed, since distutils >> provides the interfaces needed to extend its command classes >> via the setup() call. > > The monkeypatching is there so that the easy_install command can build eggs > for packages that use the distutils. It's also there so that other > distutils extensions that monkeypatch distutils (and there are a few of > them out there) will have a better chance of working with setuptools. > > I originally took a minimally-invasive approach to setuptools-distutils > interaction, but it was simply not possible to provide a high-degree of > backward and "sideward" compatibility without it. In fact, I seem to > recall finding some behaviors in some versions of distutils that can't be > modified without monkeypatching, although the details are escaping me at > this moment. That's a very vague comment. The distutils mechanism for providing your own command classes lets you take complete control over distutils if needed. What's good about it, is that this approach doesn't modify anything inside distutils at run-time, but does these modifications on a per-setup()-call basis. As for setuptools, you import the package and suddenly distutils isn't what's documented on python.org anymore. >>> As for discussion, Guido originally brought up the question here a few >>> months ago, and it's been listed in PEP 356 for a while. I've also >>> posted things related to the inclusion both here and in distutils-sig. >> >> I know, but the discussions haven't really helped much in >> getting the setuptools design compatible with standard >> distutils. > > That's because the job was already done. :) Not much of an argument, if you ask me. Some of the design decisions you made in setuptools are simply wrong IMHO and these need to be discussed in a PEP process. > The setuptools design bends > over backwards to be compatible with Python 2.3 and 2.4 versions of > distutils, not to mention py2exe, Pyrex, and other distutils extensions, > along with the quirky uses of distutils that exist in dozens of distributed > Python packages. > > However, I think you and I may perhaps have different definitions of > "compatibility". Mine is that things "just work" and users don't have to > do anything special. For that definition, setuptools is extremely > compatible with the standard distutils. In many situations it's more > compatible than the distutils themselves, in that more things "just work". ;) You've implemented your own view of "just works". This is fine, but please remember that Python is a collaborative work, so design decisions have to be worked out in collaboration as well. There aren't all that many things that are wrong in setuptools, but some of them are essential: * setuptools shoul
Re: [Python-Dev] 2.5a1 Performance
Tim Peters wrote: > [M.-A. Lemburg] >> I could contribute pybench to the Tools/ directory if that >> makes a difference: > > +1. It's frequently used and nice work. Besides, then we could > easily fiddle the tests to make Python look better ;-) That's a good argument :-) Note that the tests are versioned and the tools refuses to compare tests with different version numbers. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 18 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
On Tue, Apr 18, 2006 at 08:55:18PM +0200, phillip.eby wrote: > Modified: >peps/trunk/pep-0343.txt > > +"context manager" then encompasses all objects with a __context__() > +method that returns a context object. (This means that all contexts > +are context managers, but not all context managers are contexts). This change reminds of another question I had about the parenthetical statement: all contexts are context managers (= 'has a __context__' method). Why? The context object isn't necessarily available to the Python programmer, so they can't write: with context_mgr as context: with context: # uses the same context ... Why do contexts need to have a __context__() method? --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
"tomer filiba" <[EMAIL PROTECTED]> wrote in message >* using lambda functions for meta expressions, instead of eval(string) -- >perhaps >it's faster, but lambda is getting deprecated by python3k :( Good news for you then: Guido's latest thought that I have read is to leave lambda alone, as is. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
On 4/17/06, tomer filiba <[EMAIL PROTECTED]> wrote:
> after several people (several > 10) contacted me and said "IMHO 'construct'
> is a good candidate for stdlib",
> i thought i should give it a try. of course i'm not saying it should be
> included right now, but in 6 months time, or such a
> timeframe (aiming at python 2.6? some 2.5.x release?)
Now that ctypes is part of the standard library, that provides a
structured datatype facility. Here's an example demonstrating the
first example from the Construct wiki:
>>> from ctypes import *
>>> def str_to_ctype(s, typ):
...t = typ()
...memmove(addressof(t), s, sizeof(t))
...return t
...
>>> class ethernet_header(Structure):
... _fields_ = [("destination", c_char * 6),
... ("source", c_char * 6),
... ("type", c_short)]
...
>>> s = "ABCDEF123456\x08\x00"
>>> e = str_to_ctype(s, ethernet_header)
>>> e.source
'123456'
>>> e.destination
'ABCDEF'
>>> e.type
8
I'm not sure I understand the other wiki examples - but the ones I do,
look doable in ctypes.
There are a couple of things to note:
* ctypes doesn't have a way (that I'm aware of) to specify the
endianness of types like c_short - so my example, when run on Windows
(intel architecture) gives type = 8, rather than type = 2048 (from the
wiki). But the wiki example doesn't explicitly specify endianness, so
maybe that's a limitation in Construct as well?
* ctypes doesn't have an easy way to parse a string based on a
structure definition - hence my str_to_ctype function. But that's a
trivial helper function to write, so that's not a big issue.
Personally, I'd rather see the ctypes facilities for structure packing
and unpacking be better documented, and enhanced if necessary, rather
than having yet another way of doing the same thing added to the
stdlib.
Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Updated: PEP 359: The make statement
On 4/18/06, Steven Bethard <[EMAIL PROTECTED]> wrote: > I've updated PEP 359 with a bunch of the recent suggestions. The > patch is available at: > http://bugs.python.org/1472459 > and I've pasted the full text below. > > I've tried to be more explicit about the goals -- the make statement > is mostly syntactic sugar for:: > >class : >__metaclass__ = > > > so that you don't have to lie to your readers when you're not actually > creating a class. I've also added some new examples and expanded the > discussion of the old ones to give the statement some better > motivation. And I've expanded the Open Issues discussions to consider > a few alternate keywords and to indicate some of the difficulties in > allowing a ``__make_dict__`` attribute for customizing the dict in > which the block is executed. Guido has pronounced on this PEP: http://mail.python.org/pipermail/python-3000/2006-April/000936.html Consider it dead. =) STeVe -- Grammar am for people who can't think for myself. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
At 04:18 PM 4/18/2006 -0400, A.M. Kuchling wrote: >On Tue, Apr 18, 2006 at 08:55:18PM +0200, phillip.eby wrote: > > Modified: > >peps/trunk/pep-0343.txt > > > > +"context manager" then encompasses all objects with a __context__() > > +method that returns a context object. (This means that all contexts > > +are context managers, but not all context managers are contexts). > >This change reminds of another question I had about the parenthetical >statement: all contexts are context managers (= 'has a __context__' >method). Why? The context object isn't necessarily available to the >Python programmer, so they can't write: > >with context_mgr as context: > with context: # uses the same context > ... > >Why do contexts need to have a __context__() method? I was going to say, "so they can be context managers", but I suppose you have a point. There is no need for a context to have a __context__ method, unless it is also a context manager. Ugh. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
Paul Moore wrote:
> On 4/17/06, tomer filiba <[EMAIL PROTECTED]> wrote:
>> after several people (several > 10) contacted me and said "IMHO 'construct'
>> is a good candidate for stdlib",
>> i thought i should give it a try. of course i'm not saying it should be
>> included right now, but in 6 months time, or such a
>> timeframe (aiming at python 2.6? some 2.5.x release?)
>
> Now that ctypes is part of the standard library, that provides a
> structured datatype facility. Here's an example demonstrating the
> first example from the Construct wiki:
>
from ctypes import *
>
def str_to_ctype(s, typ):
> ...t = typ()
> ...memmove(addressof(t), s, sizeof(t))
> ...return t
> ...
class ethernet_header(Structure):
> ... _fields_ = [("destination", c_char * 6),
> ... ("source", c_char * 6),
> ... ("type", c_short)]
> ...
s = "ABCDEF123456\x08\x00"
e = str_to_ctype(s, ethernet_header)
>
e.source
> '123456'
e.destination
> 'ABCDEF'
e.type
> 8
>
> I'm not sure I understand the other wiki examples - but the ones I do,
> look doable in ctypes.
>
> There are a couple of things to note:
>
> * ctypes doesn't have a way (that I'm aware of) to specify the
> endianness of types like c_short - so my example, when run on Windows
> (intel architecture) gives type = 8, rather than type = 2048 (from the
> wiki). But the wiki example doesn't explicitly specify endianness, so
> maybe that's a limitation in Construct as well?
The currently undocumented BigEndianStructure or LittleEndianStructure
bases classes allow to specify the byte order. They are designed so
that they refuse to work when the structure contains pointer fields.
> * ctypes doesn't have an easy way to parse a string based on a
> structure definition - hence my str_to_ctype function. But that's a
> trivial helper function to write, so that's not a big issue.
>
> Personally, I'd rather see the ctypes facilities for structure packing
> and unpacking be better documented, and enhanced if necessary, rather
> than having yet another way of doing the same thing added to the
> stdlib.
It is not yet too late (but the timeslot left is very small) to propose
enhancements to ctypes. classmethods like 'from_string', 'from_buffer' or
whatever would probably make sense.
Thomas
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are contexts also managers? (was r45544 - peps/trunk/pep-0343.txt)
On Tue, Apr 18, 2006 at 03:37:37PM -0400, Phillip J. Eby wrote: > I was going to say, "so they can be context managers", but I suppose you > have a point. There is no need for a context to have a __context__ method, > unless it is also a context manager. Ugh. It would be easy to just remove the parenthetical comment from the PEP and forget about it, if in fact the statement is now purposeless. But these are murky waters, and maybe there's still some deeper reason for it. Nick, do you have any comments? --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
ctypes, as the name implies, is relevant to *C data structures* only.you cannot extend it and you cannot define complex things with it, at least noteasily.
* ctypes doesn't have a way (that I'm aware of) to specify theendianness of types like c_short - so my example, when run on Windows(intel architecture) gives type = 8, rather than type = 2048 (from thewiki). But the wiki example doesn't explicitly specify endianness, so
maybe that's a limitation in Construct as well?in ctypes, the endianness of numbers is determined by the platform, since
they are passed to a C (platform-dependent) function. you share your addressspace with the dll you load -- so both python and the dll live on the same platform.so except of writing data to files or sockets, you shouldn't care about the byte
ordering.
in Construct you have UBInt16 and ULInt16, for big and little ordering. and UInt16 is an alias to UBInt16 (because network ordering is more common in protocols)
* ctypes doesn't have an easy way to parse a string based on astructure definition - hence my str_to_ctype function. But that's atrivial helper function to write, so that's not a big issue.sorry, but if you mean people must use memmove in order to parse string,
you better slap yourself. this is a python mailing list, not a C one. we don't have a concept of addressof() or physically moving data. we use objects andreferences. no offense, but "so that's not a big issue" makes me think you
don't belong to this mailing list.I'm not sure I understand the other wiki examples - but the ones I do,
look doable in ctypes.i gues you should also look at http://pyconstruct.wikispaces.com/demos
to geta better understanding, but i only uploaded it a couple of hours ago. sorry for that. anyway, on the projects page i explain thoroughly why there is room for yet anotherparsing/building library.
but for the example you mentioned above, the ethernet header, struct is good enough:struct.pack(">6s6sH", "123456", "ABCDEF", 0x0800)
but --how would you parse a pascal-string (length byte followed by data of that length)using ctypes? how would you read a 61 bit, unaligned field? how would you convert "\x00\x11P\x88kW" to "00-11-50-88-6B-57", the way people would like to see MAC
addresses? yeah, the MAC address is only a representation issue, but adapters can do much more powerful things. plus, people usually prefer seeing "IP" instead of "0x0800" in their parsed objects. how would you define mappings in ctypes?
Personally, I'd rather see the ctypes facilities for structure packingand unpacking be better documented, and enhanced if necessary, rather
than having yet another way of doing the same thing added to thestdlib.the stdlib is too messy already. it must be revised anyway, since it's full of shitnobody uses. the point is -- ctypes can define C types. not the TCP/IP stack. Construct can do both.
it's a superset of ctype's typing mechanism. but of course both have the right to *coexist* --
ctypes is oriented at interop with dlls, and provides the mechanisms needed for that.
Construst is about data structures of all sorts and kinds. ctypes is a very helpful library as a builtin, and so is Construct. the two don't competeon a spot in the stdlib.-tomer
On 4/18/06, Paul Moore <[EMAIL PROTECTED]
> wrote:
On 4/17/06, tomer filiba <[EMAIL PROTECTED]> wrote:> after several people (several > 10) contacted me and said "IMHO 'construct'
> is a good candidate for stdlib",
> i thought i should give it a try. of course i'm not saying it should be> included right now, but in 6 months time, or such a> timeframe (aiming at python 2.6? some 2.5.x release?)Now that ctypes is part of the standard library, that provides a
structured datatype facility. Here's an example demonstrating thefirst example from the Construct wiki:>>> from ctypes import *>>> def str_to_ctype(s, typ):...t = typ()
...memmove(addressof(t), s, sizeof(t))...return t...>>> class ethernet_header(Structure):... _fields_ = [("destination", c_char * 6),... ("source", c_char * 6),
... ("type", c_short)]...>>> s = "ABCDEF123456\x08\x00">>> e = str_to_ctype(s, ethernet_header)>>> e.source'123456'>>>
e.destination
'ABCDEF'>>> e.type8I'm not sure I understand the other wiki examples - but the ones I do,look doable in ctypes.There are a couple of things to note:* ctypes doesn't have a way (that I'm aware of) to specify the
endianness of types like c_short - so my example, when run on Windows(intel architecture) gives type = 8, rather than type = 2048 (from thewiki). But the wiki example doesn't explicitly specify endianness, so
maybe that's a limitation in Construct as well?* ctypes doesn't have an easy way to parse a string based on astructure definition - hence my str_to_ctype function. But that's atrivial helper function to write, so that's not a big issue.
Personally, I'd rather see the ctypes facilities for structure packingand unpacking be better documented, and enhanced if necessary, rathertha
[Python-Dev] a flattening operator?
i'm not going to defend and fight for this idea too much. i only bringit up because it bothers me. i'm sure some people here would kill me for even suggesting this, and i really don't want to be killed right now,so i bring it up as something you should think about. nothing more.PEP-225 has some weird ideas which may or may not be related to this, but i don't understand how this magical ~ operator can do everything from tuple flattening to list arithmetics, replacing map(), changing the order of operations, deep-copying, list comprehension, rich comparison, and whatever not. so i don't consider this a serious PEP. looks more like an april fool's joke to me, and it seems those japanese celebrate it on september for some reason. [reposted from comp.lang.python]as we all know, * (asterisk) can be used to "inline" or "flatten" a tuple into an argument list, i.e.: def f(a, b, c): ... x = (1,2,3) f(*x) so... mainly for symmetry's sake, why not make a "flattening" operator that also works outside the context of function calls? for example: a = (1,2,3) b = (4,5) c = (*a, *b) # ==> (1,2,3,4,5) yeah, a + b would also give you the same result, but it could be used like format-strings, for "templating" tuples, i.e. c = (*a, 7, 8, *b) i used to have a concrete use-case for this feature some time ago, but i can't recall it now. sorry. still, the main argument is symmetry: it's a syntactic sugar, but it can be useful sometimes, so why limit it to function calls only? allowing it to be a generic operator would make things like this possible: f(*args, 7) # an implied last argument, 7, is always passed to the function today you have to do f(*(args + (7,))) which is quite ugly. and if you have to sequences, one being a list and the other being a tuple, e.g. x = [1,2] y = (3,4) you can't just x+y them. in order to concat them you'd have to use "casting" like f(*(tuple(x) + y)) instead of f(*x, *y) isn't the latter more elegant? just an idea. i'm sure people could come up with more creative use-cases of a standard "flattening operator". but even without the creative use cases -- isn't symmetry strong enough an argument? why are function calls more important than regular expressions? and the zen supports my point: (*) Beautiful is better than ugly --> f(*(args + (7,))) is ugly (*) Flat is better than nested --> less parenthesis (*) Sparse is better than dense --> less noise (*) Readability counts --> again, less noise (*) Special cases aren't special enough to break the rules --> then why are function calls so special to add a unique syntactic sugar for them? the flattening operator would work on any sequence (having __iter__ or __next__), not just tuples and lists. one very useful feature i can think of is "expanding" generators, i.e.: print xrange(10) # ==> xrange(10) print *xrange(10) # ==> (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) i mean, python already supports this half-way: >>> def f(*args): ... print args ... >>> f(*xrange(10)) (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) so... why can't i just do "print *xrange(10)" directly? defining a function just to expand a generator? well, i could use list(xrange(10))to expand it, but it's less intuitive. the other way is with list-comprehension,[x for x in xrange(10)]which is just, but isn't *xrange(10) more to-the-point? also, "There should be one-- and preferably only one --obvious way to do it"... so which one? (*) list(xrange(10)) (*) [x for x in xrange(10)] (*) mylist.extend(xrange(10)) (*) f(*xrange(10)) they all expand generators, but which is the preferable way? and imagine this: f(*xrange(10), 7) this time you can't do *(xrange(10) + (7,)) as generators do not support addition... you'd have to do *(tuple(xrange(10)) + (7,)) which is getting quite long already. so as you can see, there are many inconsistencies between function-call expressions and regular expressions, that impose artificial limitations on the language. after all, the code is already in there to support the function-call version... all it takes is adding support for regular expressions.so, what do you think? isn't symmetry worth it?-tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
On 4/18/06, Thomas Heller <[EMAIL PROTECTED]> wrote: > It is not yet too late (but the timeslot left is very small) to propose > enhancements to ctypes. classmethods like 'from_string', 'from_buffer' or > whatever would probably make sense. A from_buffer classmethod would probably be good. I didn't think to suggest it as I recall from a long time ago on the ctypes list, a discussion which basically came to the conclusion that this wasn't needed (I think it was that discussion that resulted in the inclusion of memmove etc). But if you're not against the idea, I'd go for it - it took me a bit of thinking to get my helper function right. Also, I'm not sure if a to_string method has any value. You can use str(buffer(obj)), but I don't know if that's a good long-term solution - the buffer object has always been somewhat discouraged, AFAICT. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ([Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
At 09:02 PM 4/18/2006 +0200, M.-A. Lemburg wrote: >Phillip J. Eby wrote: > > At 07:15 PM 4/18/2006 +0200, M.-A. Lemburg wrote: > >> Why should a 3rd party extension be hot-fixing the standard > >> Python distribution ? > > > > Because setuptools installs things in zip files, and older versions of > > pydoc don't work for packages zip files. > >That doesn't answer my question. That is the answer to the question you asked: "why hot-fix?" Because setuptools uses zip files, and older pydocs crash when trying to display the documentation for a package (not module; modules were fine) that is in a zip file. >Third-party extension *should not do this* ! If you install setuptools, you presumably would like for things to work, and the hot fix eliminates a bug that interferes with them working. I'm not sure, however, what you believe any of that has to do with python-checkins or python-dev. The version of setuptools that will do this is not yet released, and the hotfix aspect will be documented. If it causes actual problems for actual setuptools users, it will be actually fixed or actually removed. :) (Meanwhile, the separately-distributed setuptools package is not a part of Python, any more than the Optik is, despite Python including the 'optparse' package spun off from Optik.) >I'm talking about the setuptools package which does apply >monkey patching and is needed to manage the download and >installation of plugin eggs, AFAIK. In which case the monkeypatching is needed, to handle situations like building eggs for third-party non-setuptools libraries listed as dependencies. You can't have it both ways; either you need setuptools or you don't. If you do, you will almost certainly need the monkeypatching, and you'll definitely need it to build eggs for non-setuptools-based packages. >What's good about it, is that this approach doesn't modify anything >inside distutils at run-time, but does these modifications >on a per-setup()-call basis. What's bad about it, is that it provides no options for virtualizing the execution of other packages' setup scripts. The 'run_setup' mechanism was horribly broken in the versions of the distutils that I wrestled with. (I also don't see anybody stepping up to provide alternative implementations or patches to third-party setup scripts or distutils extensions to deal with this issue.) >There aren't all that many things that are wrong in setuptools, >but some of them are essential: > >* setuptools should not monkey patch distutils on import Please propose an alternative, or better still, produce a patch. Be sure that it still allows distutils extensions like py2exe to work. The only real alternative to monkeypatching that I'm aware of for that issue is to actually merge setuptools with the distutils, but my guess is that you'll like that even less. :) >* the standard "python setup.py install" should continue to > work as documented in the Python documentation; any new install > command should use a different name, e.g. "install_egg" > >* placing too many ZIP files on PYTHONPATH is a bad idea > since it slows down import searches for all Python > applications, not just ones relying on eggs; a solution > to this would be to have a PYTHONEGGPATH which is then > only scanned by egg-aware modules/applications > >* the user should have freedom of choice in whether to > have her Python installation rely on eggs or not (and > not only --by-using-some-complicated-options) These questions have been hashed to death on the distutils-sig already, but I will try to summarize my responses here as briefly as practical while still covering the full spectrum of issues. I would begin by saying that the tradeoffs I've made favor inexperienced users at the expense of experts (who I assume are capable of learning to use --root or --single-version-externally managed in order to have finer-grained control). Your proposals, however, generally favor experts at the expense of the average user, and treat it as axiomatic that the benefits provided by setuptools are not worth having, no matter how small the cost. By contrast, users who *want* to use setuptools for either distribution or installation of packages, would rather it "just work", no matter how much complexity is required behind-the-scenes. They don't care about my cost to implement, they just care that they type "easy_install foo" to get a package from PyPI or "setup.py sdist bdist_egg upload" to put it there. Therefore, it makes no sense to apply your design approach to setuptools, since by your criteria it wouldn't exist in the first place! After all, expert users can munge distutils extensions to their hearts' content (and you certainly have done so already). Next, your own measurements posted to the distutils-sig debunked the PYTHONPATH performance question, IMO: if you installed *every Python package listed on PyPI at that time* as a zip file (assuming that the
Re: [Python-Dev] Updated: PEP 359: The make statement
> Removing __metaclass__ in Python 3000 > - > > As a side-effect of its generality, the make-statement mostly > eliminates the need for the ``__metaclass__`` attribute in class > objects. Thus in Python 3000, instead of:: (...) One of the reasons that this PEP was born is because metaclasses are being used in ways that don't look natural to some, like generating interface instances out of a class statement, so this would add an interesting way to support these constructs. That doesn't look like a good reason, though, to kill the metaclass support that Python took so long to maturate. Otherwise, a new optional extension could be included as well: "Removing the 'class' statement". -- Gustavo Niemeyer http://niemeyer.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Updated: PEP 359: The make statement
> Consider it dead. =) RIP. ;) -- Gustavo Niemeyer http://niemeyer.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Phillip J. Eby wrote: > Your proposals, however, generally favor experts at the expense of the > average user, and treat it as axiomatic that the benefits provided by > setuptools are not worth having, no matter how small the cost. mal's arguing from well-established Python design principles (import this), and the kind of respect for existing non-core developers that has been a hallmark of Python development for as long as I've used it. who decided that setuptools should be added to 2.5, btw? it's still listed under "possible additions" in the release PEP, and there don't seem to be a PEP or any other easily located document that explains exactly how it works, what's required from library developers, how it affects existing toolchains, etc. is this really ready for inclusion ? does anyone but Phillip understand how it works ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a flattening operator?
"tomer filiba" <[EMAIL PROTECTED]> wrote: > isn't the latter more elegant? According to my experience with Python, as well as my interpretations of the zens, no. -1 > and the zen supports my point: > (*) Beautiful is better than ugly --> f(*(args + (7,))) is ugly But explicit is better than implicit, and in this case, it would likely be better to just use a keyword argument lastarg=7. If the function called merely accepts *args and **kwargs, and you are passing args unchanged somewhere else, it may be more explicit to redefine f to be... def f(args, kwargs): ... Then use it like... f(args + (7,)) Not quite as ugly as the worst-case option were you were describing moments ago. > (*) Flat is better than nested --> less parenthesis I believe that zen was in regards to overly-nested object hierarchies and namespaces. > (*) Sparse is better than dense --> less noise I believe that zen was in regards to using spaces between operations, as well as using line breaks between sections of code so as to express a visual 'separating' of ideas. > (*) Readability counts --> again, less noise Use keyword arguments for single arguments, ... > (*) Special cases aren't special enough to break the rules --> then why > are function calls so special to add a unique syntactic sugar for them? By necessity, function defintions require a unique syntax to make calls different from anything else you can do with an object. Specifically, we have f() because () are unambiguous for calling functions due to various reasons. > the flattening operator would work on any sequence (having __iter__ or > __next__), not just tuples and lists. one very useful feature i can > think of is "expanding" generators, i.e.: > > print xrange(10) # ==> xrange(10) > print *xrange(10) # ==> (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) One major problem with this is that except for function calls, * is the multiplication operator, which operates on two arguments. *foo is an operation on a single argument, and without parenthesis, would be ambiguously parsed. Further, I would prefer not to see this monstrosity: print x*(*y) Which would print out x copies of the sequence y. With all of that said, there is a recent discussion in the python 3k mailing list which has been discussing "Cleaning up argument list parsing". The vast majority of your ideas can be seen as a variant of the discussion going on there. I haven't been paying much attention to it mostly because I think that the current convention is sufficient, and allowing people to do things like... foo(*arglist1, b, c, d=d_arg, *arglist2, **kwargs) ...is just begging for a mess. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45505 - python/trunk/Modules/posixmodule.c
[EMAIL PROTECTED] wrote: > Martin> Also, I suggest to use None as the return value for "no value > Martin> available"; it might be that the configured value is an empty > Martin> string (in which case confstr returns 1). > > I'll work on all of this. Are you sure you want the API to change? Wrt. to the "no configured value" case? If everybody can agree it is the conceptually right thing to do (*), then sure; documentation should get updated, of course (if there is any). This was so broken already that I'm not worried about breaking some user's code: all users apparently only ever used the "successful" cases. OTOH, if people debate whether this actually is the right thing to do, it should not change. Regards, Martin (*) I believe it is conceptually right, because it allows to distinguish two cases which are currently indistinguishable in Python but distinguishable in C, namely: a) there is no configured value (confstr returns 0 an does not change errno), and b) the configured value is an empty string (confstr returns 1). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Fredrik Lundh wrote: > it's still listed under "possible additions" in the release PEP, and there > don't > seem to be a PEP or any other easily located document that explains exactly > how it works, what's required from library developers, how it affects existing > toolchains, etc. is this really ready for inclusion ? does anyone but > Phillip > understand how it works ? I don't understand it. My concern is that it appears to involve a lot of magic. This magic might do the "right thing" in many cases, and it might indeed help the user that the magic is present, yet I still do believe that magic is inherently evil: explicit is better than implicit. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
At 11:55 PM 4/18/2006 +0200, Fredrik Lundh wrote: >who decided that setuptools should be added to 2.5, btw? Guido proposed it on Python-dev when the 2.5 schedule was first being discussed. I discussed it with him off-list, to ensure that it could be done in a way that wouldn't interfere with existing setuptools users or affect Python itself in a negative way. (For example, it needed to be upgradeable in the field in case users wanted/needed a later version than the one included in 2.5.) He then mentioned it in his 2.5 slideshow at PyCon. This is the first anyone's objected to it, however, at least that I'm aware of. >it's still listed under "possible additions" in the release PEP, I imagine that might be why nobody raised any objections sooner, although I understood the possibility to mean "if nobody objects". :) I also posted on Python-dev repeatedly in recent weeks, referring to how the various PEP 302 fixes and updates would interact with setuptools when it got in for 2.5. Also, Neal emailed me the week before last, asking when I would be getting setuptools checked in, and I told him April 17th - i.e., yesterday. So, I was under the impression this was all a done deal. >and there don't >seem to be a PEP or any other easily located document that explains exactly >how it works, what's required from library developers, how it affects existing >toolchains, etc. The setuptools manual is currently at: http://peak.telecommunity.com/DevCenter/setuptools pending conversion to the standard Pythondoc format. I posted earlier today asking about how it, and the other related manuals should be included in the overall Python documentation structure: http://mail.python.org/pipermail/python-dev/2006-April/063846.html The reST source of these manuals is in trunk/sandbox/setuptools, where it has been evolving over the last year. > is this really ready for inclusion ? Please define "ready". I don't mean that in a flippant way, I just don't know what you mean. >does anyone but Phillip understand how it works ? Does anybody besides Thomas understand how ctypes works? ;) Rhetorical jokes aside, every time I've made a significant change to how setuptools works, I've posted it to the distutils-sig -- and usually I make proposals in advance to get feedback or stimulate discussion. I regularly post explanations there in response to questions from people who are integrating it with system packaging tools, or creating various other customizations. And there are other people on distutils-sig who can answer questions about it. The TurboGears community is proficient enough with it that it's only once every few months now that a question gets kicked upstairs to me to answer. A number of people have contributed patches, including Ian Bicking and Tres Seaver. Bob Ippolito was a significant participant in the original design and wrote some of the initial code for the runtime. A *considerable* number of distutils-sig participants have had design input, either through direct suggestions, or through their giving more use case examples that I needed to make "just work". So, I'm not too pleased by insinuations that setuptools is anything other than a Python community project. But MAL and MvL are the only folks from Python-Dev who I've seen over there arguing for changes to setuptools -- and I actually made changes based on their input, although they rarely got 100% of what they asked for. The --really-long-option MAL is complaining about was put in to provide a feature that *he and MvL wanted me to include*; I just don't want that behavior to be the default behavior for setuptools. (And neither do package developers who have to support "non-root" users on virtual hosting systems, or other environments where system packaging tools aren't available.) So, it seems to me that MAL claiming that nobody got to participate in the design process is rather misleading. It's like somebody who wanted decorators in Python and then gripes about the '@' syntax. Everybody's got to compromise a bit. I put their feature in months ago, and it is even the default when you use --root with install. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib
On Tuesday 18 April 2006 19:00, Phillip J. Eby wrote: > He then mentioned it in his 2.5 slideshow at PyCon. This is the first > anyone's objected to it, however, at least that I'm aware of. Until the past week, I wasn't aware it was being considered. But then, I've not been paying a lot of attention lately, so I suspect that's my fault. > The setuptools manual is currently at: > > http://peak.telecommunity.com/DevCenter/setuptools > > pending conversion to the standard Pythondoc format. I posted earlier > today asking about how it, and the other related manuals should be > included in the overall Python documentation structure: Saw that; hopefully I'll have a chance to look at it soon. I wonder, generally, if it should be merged into the distutils documentation. Those documents happen to be distutils-centric now, because that's what's been provided. Their titles should be the guide to their content, however. > So, I'm not too pleased by insinuations that setuptools is anything other > than a Python community project. I've no doubt about that at all, FWIW. I think you've put a lot of effort into discussing it with the community, and applaud you for that as well as your implementation efforts. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
tomer filiba <[EMAIL PROTECTED]> wrote: > the point is -- ctypes can define C types. not the TCP/IP stack. > Construct can do both. it's a superset of ctype's typing mechanism. > but of course both have the right to *coexist* -- > ctypes is oriented at interop with dlls, and provides the mechanisms > needed for that. > Construst is about data structures of all sorts and kinds. > > ctypes is a very helpful library as a builtin, and so is Construct. > the two don't compete on a spot in the stdlib. I don't agree. Both ctypes and construct provide a way to describe a binary-packed structure in Python terms: and this is an overload of functionality. When I first saw Construct, the thing that crossed my head was: "hey, yet another syntax to describe a binary-packed structure in Python". ctypes uses its description to interoperate with native libraries, while Construct uses its to interoperate with binary protocols. I didn't see a good reason why you shouldn't extend ctypes so to provide features that it is currently missing. It looks like it could be easily extended to do so. Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
At 12:49 AM 4/19/2006 +0200, Martin v. Löwis wrote: >Fredrik Lundh wrote: > > it's still listed under "possible additions" in the release PEP, and > there don't > > seem to be a PEP or any other easily located document that explains exactly > > how it works, what's required from library developers, how it affects > existing > > toolchains, etc. is this really ready for inclusion ? does anyone but > Phillip > > understand how it works ? > >I don't understand it. Have you read the manuals? > My concern is that it appears to involve a lot of >magic. Please define "magic". Better, please point to the API functions or classes, or the setup commands documented in the manual, to show me what these things are that appear to be magic. There do exist implementation details that are not included in user documentation, but this is going to be true of any system. These details are sometimes necessarily complex due to distutils limitations, behavioral quirks of deployed packages using distutils, and the sometimes baroque variations in sys.path resolution across platforms, Python versions, and invocation methods. That these details are not discussed at length in the user documentation is because they are boring hideous things that I wish I didn't have to know myself, and that few people using setuptools would ever want to know. They *do*, however, get discussed (or at least explained by me) at length on the distutils-sig. I explain the quirks and the tradeoffs for each bit of implementation in detail there, and have been doing so for nearly a year now. Anybody who wants to know what's going on has had plenty of opportunity to learn. >This magic might do the "right thing" in many cases, and it might >indeed help the user that the magic is present, yet I still do believe >that magic is inherently evil: explicit is better than implicit. Are documented defaults "implicit" or "magic"? To the extent that there is anything that may be called "magic" in setuptools, it exists only because the necessary infrastructure isn't already present in Python, or because it was required to work around the quirks of other systems. Setuptools chooses to "work at all costs" because backward-compatibility skirts the chicken-and-egg problems that would otherwise exist. If setuptools *weren't* so highly backward-compatible, its use would never have caught on enough to allow this discussion to be taking place in the first place. :) The primary place where it *isn't* backward compatible is if somebody does "setup.py install" *without* using --root, *and* they care about the exact file layout of the result, *and* they are installing a package that explicitly uses setuptools... in which case their complaint is with the author of the package, if the author didn't explain that they used setuptools or point them to relevant documentation. Setuptools' manual prominently explains what developers should tell their users, if they use setuptools in their setup.py: """To keep these users happy, you should review the following topics in your project's installation instructions, if they are relevant to your project and your target audience isn't already familiar with setuptools and easy_install.""" http://peak.telecommunity.com/DevCenter/setuptools#what-your-users-should-know With respect to you and MAL, I think that some of your judgments regarding setuptools may have perhaps been largely formed at a time last year when, among other things: * That documentation section I just referenced didn't exist yet * Many common installation scenarios (e.g. custom PYTHONPATH) didn't "just work" without special setup steps * --single-version-externally-managed was barely a proposal, and it wasn't automatically activated when --root was used These are significant changes that are directly relevant to the objections that you and he raised (regarding startup time, path length, tools compatibility, etc.), and which I added because of those objections. I think that they are appropriate responses to the issues raised, in that they allow the audience that cares about them (you, MAL, and system packagers in general) to avoid the problems that those features' absence caused. It would probably helpful if you would both be as specific as possible in your objections so that they can be addressed individually. If you don't want setuptools in 2.5, let's be clear either on the specific objections. If the objection is to the very *idea* of setuptools, that's fine too, as long as we're clear that's what the objection is. So I would ask that you please make a list of what you would have to see changed in setuptools before you would consider it viable for stdlib inclusion, or simply admit that there are no changes that would satisfy you, or that you don't know enough about it to say, or that you'd like it to be kicked back to distutils-sig for more discussion ad infinitum, or whatever your actual objections
Re: [Python-Dev] setuptools in the stdlib
At 07:08 PM 4/18/2006 -0400, Fred L. Drake, Jr. wrote: >Saw that; hopefully I'll have a chance to look at it soon. I wonder, >generally, if it should be merged into the distutils documentation. Those >documents happen to be distutils-centric now, because that's what's been >provided. Their titles should be the guide to their content, however. No doubt that's the proper thing to do in the long term, when/if setuptools is the official One Obvious Way To Do It. I was wondering, however, if perhaps Python 2.5 should include them as "Building and Distributing Python Eggs" (for what's now setuptools.txt) and "Installing Python Eggs" (for what's now EasyInstall.txt). >I've no doubt about that at all, FWIW. I think you've put a lot of effort >into discussing it with the community, and applaud you for that as well as >your implementation efforts. Thank you. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
Giovanni Bajo wrote: > tomer filiba <[EMAIL PROTECTED]> wrote: > > >>the point is -- ctypes can define C types. not the TCP/IP stack. >>Construct can do both. it's a superset of ctype's typing mechanism. >>but of course both have the right to *coexist* -- >>ctypes is oriented at interop with dlls, and provides the mechanisms >>needed for that. >>Construst is about data structures of all sorts and kinds. >> >>ctypes is a very helpful library as a builtin, and so is Construct. >>the two don't compete on a spot in the stdlib. > > > > I don't agree. Both ctypes and construct provide a way to describe a > binary-packed structure in Python terms: and this is an overload of > functionality. When I first saw Construct, the thing that crossed my head was: > "hey, yet another syntax to describe a binary-packed structure in Python". > ctypes uses its description to interoperate with native libraries, while > Construct uses its to interoperate with binary protocols. I didn't see a good > reason why you shouldn't extend ctypes so to provide features that it is > currently missing. It looks like it could be easily extended to do so. > For what it's worth, NumPy also defines a data-type object which it uses to describe the fundamental data-type of an array. In the context of this thread it is also yet another way to describe a binary-packed structure in Python. This data-type object is a builtin object which provides information such as byte-order, element size, "kind" as well as the notion of fields so that nested structures can be easily defined. Soon (over the next six months) a basic array object (a super class of NumPy) will be proposed for inclusion in Python. When that happens some kind of data-type object (a super class of the NumPy dtype object) will be needed as well. I think some cross-talk between all of us different users of the notion of what we in the NumPy community call a data-type might be useful. -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib
Fred L. Drake, Jr. <[EMAIL PROTECTED]> wrote: > > So, I'm not too pleased by insinuations that setuptools is > anything other > than a Python community project. > > I've no doubt about that at all, FWIW. I think you've put a lot of > effort into discussing it with the community, and applaud you for > that as well as your implementation efforts. I agree but I have a question for Phil though: why can't many of the setuptools feature be simply integrated within the existing distutils? I have been fighting with distutils quite some time and have had to monkey-patch it somehow to fit my needs. I later discovered that setuptools included many of those fixes already (let alone the new features). I actually welcome all those setuptools fixes in the "Just Works(TM)" principle with which I totally agree. But, why can't setuptools be gradually merged into distutils, instead of being kept as a separate package? Let's take a real example: setuptools' sdist is much enhanced, has integration with CVS/SVN, uses MANIFEST in a way that it really works, etc. Why can't it be merged into the original distutils? Is it just for backward compatibility? If so, can't we have some kind of versioning system? Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib
At 02:12 AM 4/19/2006 +0200, Giovanni Bajo wrote: >But, why can't setuptools be gradually merged into distutils, instead of being >kept as a separate package? Let's take a real example: setuptools' sdist is >much enhanced, has integration with CVS/SVN, uses MANIFEST in a way that it >really works, etc. Why can't it be merged into the original distutils? Is it >just for backward compatibility? This specific issue was discussed last year on the distutils-sig, and the issue is indeed one of compatibility. Setuptools' behavior for MANIFEST generation definitely matches new or infrequent users' expectations 1000% better than the distutils, and requires much less work to get right, even for experts. But for anybody who has extended the distutils using external tools, it would not necessarily work. MAL gave the example of someone who has written other scripts or Makefile rules to add things to MANIFEST or use it to do other things. They might be relying on quirks of the existing behaviors, in other words, and thus it should not be changed without explicit action on their part. And I agree with his reasoning, although I also think that any "distutils 2" should have only One Obvious Way to send input to that process, and it should be via MANIFEST.in, not MANIFEST. Likewise, it should have only one way to get the output. However, unless somebody explicitly chooses to use "distutils 2", they should get the old behavior. This unfortunately means that we can't backport most of setuptools' enhancements to the existing distutils without breaking backward compatibility for people who may have made extensive investment in integrating with the distutils. (Of course, how many of these people exist I don't know; in my personal experience it seems rare for people to integrate with external tools in this fashion, versus simply subclassing things in Python or abandoning distutils altogether. But that's a separate question.) >If so, can't we have some kind of versioning >system? We do: "import setuptools". We could perhaps rename it to "import distutils2" if you prefer, but it would mean essentially the same thing. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Raising objections (was: setuptools in the stdlib)
On 4/18/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: > Phillip J. Eby wrote: > > As for discussion, Guido originally brought up the question here a few > > months ago, and it's been listed in PEP 356 for a while. I've also > > posted things related to the inclusion both here and in distutils-sig. > > I know, but the discussions haven't really helped much in > getting the setuptools design compatible with standard > distutils. I'm glad to see the discussions taking place; better late than never. However, I think we need to do a better job raising objections earlier. I'm not sure how to do this, but one way is to use PEPs. There is an outstanding issues section in the 2.5 release PEP 356. In this case, perhaps it would have been good to add a bullet item there. I've been trying to ensure the issues aren't lost. There's only one item in the list that still needs addressing (Fred, you listening? You had an idea for solving the problem). I plan to start a 2.6 release schedule PEP soon (before 2.5 is released). It will mostly be a place holder/boilerplate until it can be filled out when a major feature is implemented, a PEP accepted, or outstanding issue raised. The PEP should be updated by a committer when any of those occur. If anyone has better ideas how to ensure the issues aren't lost and are addressed, I'd like to know. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a flattening operator?
Josiah Carlson wrote: > One major problem with this is that except for function calls, * is the > multiplication operator, which operates on two arguments. *foo is an > operation on a single argument, and without parenthesis, would be > ambiguously parsed. No, it wouldn't. There's no problem in giving an operator different unary and binary meanings; '-' already does that. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
I'm not sure how people would prefer this be handled. I don't think we need to have a PEP for it - I don't see PEPs for ctypes, elementtree, pysqlite or cProfile, either. I don't have a problem at all with setuptools going into the standard library. It adds a whole pile of extremely useful functionality (easy_install, in particular, is something that people have been asking for, constantly, for YEARS). Making it an additional install is just silly. Sure, it's possible that some people with extremely complicated distutils scripts may find they need to update them. But the alternative to that is complete paralysis - and I can't say that the current state of distutils is at all something to make Python happy. I started refactoring some of the ugliness out of the internals of distutils last year, but was completely stymied by the demand that no existing setup.py scripts be broken. This means that the people who are experts with the current code are fine, but everyone else has to pay the price. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
On 4/18/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 11:55 PM 4/18/2006 +0200, Fredrik Lundh wrote: > >who decided that setuptools should be added to 2.5, btw? > > Guido proposed it on Python-dev when the 2.5 schedule was first being > discussed. I discussed it with him off-list, ... I thought more was discussed on-list, but apparently not. I searched my mail and could find no direct discussion on python-dev. I saw quite a few references to setuptools being included in 2.5, but nothing explicit. That's unfortunate. I think I talked to Guido and probably Anthony off-list to see if they objected to setuptools in 2.5. My only mail to Phillip was to see if it was going in. > >it's still listed under "possible additions" in the release PEP, > > I imagine that might be why nobody raised any objections sooner, although I > understood the possibility to mean "if nobody objects". :) I was also working under the assumption that people would complain if they didn't like something. What do people think should happen for the "Possible features" section? Should I ask if there are any objections to each item? In the current list of possible features, I think the only new features that stand a chance in 2.5 are: wsgiref and python pgen. Ronald is working on the fat Mac binaries, it's only in this section because it's not complete, but anticipated. All the other things are small cleanups (icons, Demo, file/open) or things that have to happen (ssize_t cleanup). @decorator and functools are possible, but no one is doing anything about them, so I suspect they will not go in unless someone cares enough to do the work and Guido agrees. Since he's still on vacation, these won't be decided for another week or so. Probably these will be decided after a2 goes out (hopefully around the 25th). If you have any questions about the status of 2.5, speak up. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
On Wed, 2006-04-19 at 14:57 +1000, Anthony Baxter wrote: > I'm not sure how people would prefer this be handled. I don't think we > need to have a PEP for it - I don't see PEPs for ctypes, elementtree, > pysqlite or cProfile, either. Agreed. If modules like these have a solid history of use outside the stdlib I don't think we need all the formality of a PEP to pull them in. I /do/ think however that we need to be diligent in documenting them so that people who don't follow python-dev (or the packages own development forums) will become aware of what they are and how to use them. Correct me if I'm wrong, but I don't think any of the above are currently documented in the stdlib. > I don't have a problem at all with setuptools going into the standard > library. It adds a whole pile of extremely useful functionality > (easy_install, in particular, is something that people have been > asking for, constantly, for YEARS). Making it an additional install > is just silly. I agree. My one stupid nit is that I don't like the name 'easy_install'. I wish a better, non-underscored word could be found. But as I've been a total bystander in setuptools development, I have no real standing to complain. ;) > I started refactoring some of the ugliness out of the internals of > distutils last year, but was completely stymied by the demand that no > existing setup.py scripts be broken. This means that the people who > are experts with the current code are fine, but everyone else has to > pay the price. I've written some nasty setup.py scripts and I for one would be all for breaking them and rewriting them if they could be done much more simply and were better integrated with external tools. Heck, I wouldn't even mind a big ol' "if sys.hexversion" at the top of them for backward compatibility for a while if necessary. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] adding Construct to the standard library?
Travis Oliphant wrote: > For what it's worth, NumPy also defines a data-type object which it > uses to describe the fundamental data-type of an array. In the context > of this thread it is also yet another way to describe a binary-packed > structure in Python. Maybe there should be a separate module providing a data-packing facility that ctypes, NumPy, etc. can all use (perhaps with their own domain-specific extensions to it). It does seem rather silly to have about 3 or 4 different incompatible ways to do almost exactly the same thing (struct, ctypes, NumPy and now Construct). -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
At 02:57 PM 4/19/2006 +1000, Anthony Baxter wrote: >Sure, it's possible that some people with extremely complicated >distutils scripts may find they need to update them. ...if and *only* if they want setuptools' features, or their users do. Sorry to seize on this point out of context, Anthony. I just want to prevent anybody from seizing on this as an objection on "backward compatibility" grounds. Nobody will be forced to use setuptools in 2.5, any more than anyone will be forced to use ctypes or elementtree or sqlite. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
On 4/18/06, Barry Warsaw <[EMAIL PROTECTED]> wrote: > On Wed, 2006-04-19 at 14:57 +1000, Anthony Baxter wrote: > > I'm not sure how people would prefer this be handled. I don't think we > > need to have a PEP for it - I don't see PEPs for ctypes, elementtree, > > pysqlite or cProfile, either. > > Correct > me if I'm wrong, but I don't think any of the above are currently > documented in the stdlib. Ok, I will. :-) cProfile is documented in libprofile.tex. None of the others are documented, also msilib doesn't appear to have any doc. I've updated the PEP to mention these issues. Thanks for pointing it out! Note that there is probably a bunch of doc, but it all needs to be incorporated into the python repo and format. What are the doc plans for these modules: + * ctypes + * ElementTree/cElementTree + * msilib + * pysqlite n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
At 10:14 PM 4/18/2006 -0700, Neal Norwitz wrote: >On 4/18/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > > At 11:55 PM 4/18/2006 +0200, Fredrik Lundh wrote: > > >who decided that setuptools should be added to 2.5, btw? > > > > Guido proposed it on Python-dev when the 2.5 schedule was first being > > discussed. I discussed it with him off-list, ... > >I thought more was discussed on-list, but apparently not. I searched >my mail and could find no direct discussion on python-dev. I saw >quite a few references to setuptools being included in 2.5, but >nothing explicit. That's unfortunate. Here are the threads that Guido started; the longer one includes a number of discussions about setuptools-related features. MvL raised an objection to the whole idea of a Python-specific packaging format, which I responded to, and there was some other discussion about the relative utility of setuptools' approach versus learning (and building) a half-dozen bdist_* formats for different platforms: http://mail.python.org/pipermail/python-dev/2006-February/060723.html http://mail.python.org/pipermail/python-dev/2006-February/060869.html >I was also working under the assumption that people would complain if >they didn't like something. What do people think should happen for >the "Possible features" section? Should I ask if there are any >objections to each item? Well, Guido asked about including setuptools, and it doesn't seem to have done any good in this case. :) I'm not sure how much more explicit you can get. Yes, he called it setuplib in the first thread, but Georg knew what he meant anyway, and then Guido corrected himself in the second thread -- which also had a title that should've caught the eye of anybody interested in distutils-related things. (bdist_* to stdlib.) I was surprised that MAL didn't comment *then*, actually, and mistakenly thought it meant that our last discussion on the distutils-sig (and my attempts to deal with the problems) had been successful. Between that and MvL's mild response to the explicit discussion of supporting setuptools, I thought their votes had effectively moved from -1 to -0. Off-list discussion with Fredrik suggested that he too had shifted from -1 to -0, and since those were the only core developers that I knew of who had ever said anything the least bit negative about setuptools, I assumed this meant that Guido's motion had carried, so to speak. I mention this mainly to clarify that this was not some attempt by me to slip setuptools in past opposition; I genuinely thought the objectors had gone from "don't bring that stuff anywhere near me" to "I don't like it but other people seem to, so whatever." ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
At 01:33 AM 4/19/2006 -0400, Barry Warsaw wrote: >On Wed, 2006-04-19 at 14:57 +1000, Anthony Baxter wrote: > > I'm not sure how people would prefer this be handled. I don't think we > > need to have a PEP for it - I don't see PEPs for ctypes, elementtree, > > pysqlite or cProfile, either. > >Agreed. If modules like these have a solid history of use outside the >stdlib I don't think we need all the formality of a PEP to pull them in. >I /do/ think however that we need to be diligent in documenting them so >that people who don't follow python-dev (or the packages own development >forums) will become aware of what they are and how to use them. Correct >me if I'm wrong, but I don't think any of the above are currently >documented in the stdlib. I thought that ctypes doc had been added, but I guess they're still in-progress. The setuptools docs are definitely on my plan for conversion to Pythondoc format, as per my earlier post today asking where they should go in the overall doc layout. > > I don't have a problem at all with setuptools going into the standard > > library. It adds a whole pile of extremely useful functionality > > (easy_install, in particular, is something that people have been > > asking for, constantly, for YEARS). Making it an additional install > > is just silly. > >I agree. My one stupid nit is that I don't like the name >'easy_install'. I wish a better, non-underscored word could be found. The long term plan is for a tool called "nest" to be offered, which will offer a command-line interface similar to that of the "yum" package manager, with commands to list, uninstall, upgrade, and perform other management functions on installed packages. It's not likely to be available in time for the Python 2.5, release, but when it *is* available you'll just "python -m easy_install --upgrade setuptools" to get it. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib
Phillip J. Eby <[EMAIL PROTECTED]> wrote: >> If so, can't we have some kind of versioning >> system? > > We do: "import setuptools". We could perhaps rename it to "import > distutils2" if you prefer, but it would mean essentially the same > thing. :) I believe the naming is important, though. I'd rather it be called distutils2, or "from distutils.core import setup2" or something like that. setuptools *is* a new version of distutils, so it shouldn't have a different name. Then, about new commands. Why should I need to do "import distutils2" to do, eg, "setup.py develop"? This doesn't break backward compatibility. Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
Anthony Baxter wrote: > I'm not sure how people would prefer this be handled. I don't think we > need to have a PEP for it - I don't see PEPs for ctypes, elementtree, > pysqlite or cProfile, either. If I'm not calling shared libraries from Python I can ignore ctypes. If I'm not doing XML, I can ignore elementtree. If I'm not doing SQL I can ignore pysqlite and if I'm not interested in profiling I can ignore cProfile. But setuptools will potentially affect anyone that uses third-party modules/packages. And ctypes, elementtree and pysqlite are mature packages. setuptools isn't even finished yet. > I don't have a problem at all with setuptools going into the standard > library. It adds a whole pile of extremely useful functionality > (easy_install, in particular, is something that people have been > asking for, constantly, for YEARS). Making it an additional install > is just silly > > Sure, it's possible that some people with extremely complicated > distutils scripts may find they need to update them. Wouldn't I need at least have to change "from distutils.core import setup" to "from setuptools import setup"? Or to something like: try: import ez_setup except ImportError: import distutils.core as tools else: ez_setup.use_setuptools() import setuptools as tools for backwards compatibility reasons? > But the > alternative to that is complete paralysis - and I can't say that the > current state of distutils is at all something to make Python happy. > > I started refactoring some of the ugliness out of the internals of > distutils last year, but was completely stymied by the demand that no > existing setup.py scripts be broken. This means that the people who > are experts with the current code are fine, but everyone else has to > pay the price. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib
At 08:11 AM 4/19/2006 +0200, Giovanni Bajo wrote: >Then, about new commands. Why should I need to do "import distutils2" to do, >eg, "setup.py develop"? This doesn't break backward compatibility. The develop command uses the egg_info command. egg_info uses the setuptools enhanced MANIFEST scheme. Both make use of extended setup() arguments, and the entry points feature that allows distutils plugins to co-operate. Develop also uses easy_install... and so on. I'm not saying it would be impossible to merge this stuff into the distutils, just that it's not a trivial undertaking. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
Anthony Baxter wrote: > I started refactoring some of the ugliness out of the internals of > distutils last year, but was completely stymied by the demand that no > existing setup.py scripts be broken. Instead of trying to fix distutils, maybe it would be better to start afresh with a new package and a new name, then there wouldn't be any backwards-compatibility issues to hold it back. I'd like to see a different approach taken to the design altogether, something more along the lines of Scons. Maybe it could even be an extension of Scons. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 -python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Neal Norwitz wrote: > I was also working under the assumption that people would complain if > they didn't like something. What do people think should happen for > the "Possible features" section? Should I ask if there are any > objections to each item? some discussion on python-dev for each non-trivial item should be required. larger items may need discussion + grace period + discussion before they've checked in. the increasing amount of "but I've discussed this in some other forum" worries me. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Phillip J. Eby wrote: > I was surprised that MAL didn't comment *then*, actually, and mistakenly > thought it meant that our last discussion on the distutils-sig (and my > attempts to deal with the problems) had been successful. Between that and > MvL's mild response to the explicit discussion of supporting setuptools, I > thought their votes had effectively moved from -1 to -0. Off-list > discussion with Fredrik suggested that he too had shifted from -1 to -0, I'm +1 on adding stuff to distutils (and other install tools) to make it *easier* for setuptools (and other install tools) to make a good job. I'm -1 on adding tools to the core that changes the structure of an installed Python system, without a full PEP process. If nobody can point to (or produce) a technical document that, in full detail, describes the mechanisms *used* by setuptools, including what files it creates, what the files contain, how they are used during import, how non-setuptools code can manipulate (or at least in- spect) the data, etc, setuptools should *not* go into 2.5. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
On Wednesday 19 April 2006 16:22, Walter Dörwald wrote: > If I'm not calling shared libraries from Python I can ignore > ctypes. If I'm not doing XML, I can ignore elementtree. If I'm not > doing SQL I can ignore pysqlite and if I'm not interested in > profiling I can ignore cProfile. But setuptools will potentially > affect anyone that uses third-party modules/packages. Sure. It might mean people can automatically install something like TurboGears and all it's dependencies, without stuffing their existing required versions (thanks to eggs) and without having to fetch 16 different packages. Oh, the horror! > And ctypes, elementtree and pysqlite are mature packages. > setuptools isn't even finished yet. Neither is distutils. setuptools at least is likely to be "finished", whatever that means. > Wouldn't I need at least have to change "from distutils.core import > setup" to "from setuptools import setup"? Or to something like: Nope, only if you want to use the new, nicer functionality. If you want to stick with the status quo, you're quite welcome to. Anthony -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
At 08:22 AM 4/19/2006 +0200, Walter Dörwald wrote: >Anthony Baxter wrote: > > > I'm not sure how people would prefer this be handled. I don't think we > > need to have a PEP for it - I don't see PEPs for ctypes, elementtree, > > pysqlite or cProfile, either. > >If I'm not calling shared libraries from Python I can ignore ctypes. If >I'm not doing XML, I can ignore elementtree. If I'm not doing SQL I can >ignore pysqlite and if I'm not interested in profiling I can ignore >cProfile. And if you're not using setuptools, or any packages that do, you can ignore it as well. >But setuptools will potentially affect anyone that uses >third-party modules/packages. Setuptools already exists, so it already affects "anyone that uses third-party modules/packages". That genie isn't going back in the metaphorical bottle. However, if setuptools is in the stdlib, it means that those people won't have to install setuptools *first* in order to use a package that's distributed using setuptools. Thus, having it in the stdlib arguably *reduces* the impact of setuptools on such users. > > Sure, it's possible that some people with extremely complicated > > distutils scripts may find they need to update them. > >Wouldn't I need at least have to change "from distutils.core import >setup" to "from setuptools import setup"? Or to something like: > >try: > import ez_setup >except ImportError: > import distutils.core as tools >else: > ez_setup.use_setuptools() > import setuptools as tools > >for backwards compatibility reasons? If, and *only* if, you want to use setuptools, or your users hound you enough to make you. :) Nobody is *forced* to use setuptools for a package they are distributing, any more than they are forced to use ctypes. The "setuptools takes over everything" argument is purest FUD and nonsense. Developers use setuptools because they want the features, or because their users want them to. It's true that sometimes users demand setuptools support, when all they really want is the ability to build eggs. For example, some folks were bugging Greg Ewing to add it to Pyrex, and I had to point out that as long as Pyrex has a PyPI entry and a vanilla setup.py, there is no need for him to start using setuptools. So he created a PyPI entry, making it now possible for easy_install users to do this: $ python2.5 -m easy_install Pyrex Searching for Pyrex Reading http://www.python.org/pypi/Pyrex/ Reading http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ Best match: Pyrex 0.9.4 Downloading http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/Pyrex-0.9.4.tar.gz Processing Pyrex-0.9.4.tar.gz ... zip_safe flag not set; analyzing archive contents... ... Adding Pyrex 0.9.4 to easy-install.pth file Installing pyrexc script to /home/pje/bin Installed /home/pje/lib/python2.5/site-packages/Pyrex-0.9.4-py2.5.egg Processing dependencies for Pyrex $ So, nobody was forced to import setuptools in order to offer any features to end users. Setuptools is perfectly capable of running setup scripts and making them into eggs on its own, within reasonable levels of distutils customization. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a flattening operator?
Greg Ewing <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > > One major problem with this is that except for function calls, * is the > > multiplication operator, which operates on two arguments. *foo is an > > operation on a single argument, and without parenthesis, would be > > ambiguously parsed. > > No, it wouldn't. There's no problem in giving an operator > different unary and binary meanings; '-' already does > that. I stand corrected. Though the look of 'print *e' bothers me. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections (was: setuptools in the stdlib)
Phillip J. Eby wrote: > The long term plan is for a tool called "nest" to be offered, which will > offer a command-line interface similar to that of the "yum" package > manager, with commands to list, uninstall, upgrade, and perform other > management functions on installed packages. yum already exists, of course. along with many other package managers. do you expect linux and bsd packagers to switch to your stuff for all their python needs, or are you building a parallel universe ? if so, why ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
