Re: [Python-Dev] PEP 366 - Relative imports from main modules
Josiah Carlson wrote:
> "Brett Cannon" <[EMAIL PROTECTED]> wrote:
>> On 7/5/07, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
>>> At 11:53 AM 7/5/2007 +0200, Guido van Rossum wrote:
I see no big problems with this, except I wonder if in the end it
wouldn't be better to *always* define __package_name__ instead of only
when it's in main? And then perhaps rename it to __package__? Done
properly it could always be used for relative imports, rather than
parsing __module__ to find the package. Then you won't even need the
error handler.
>>> +1 for __package__, and putting it everywhere. Relative import
>>> should use it first if present, falling back to use of __name__.
>> +1 from me as well.
>
> This would solve some issues I'm currently having with relative imports.
> +1
I've updated the PEP to incorporate the feedback from this thread. The
new version is below, and should show up on the website shortly.
Cheers,
Nick.
PEP: 366
Title: Main module explicit relative imports
Version: $Revision: 56190 $
Last-Modified: $Date: 2007-07-08 17:45:46 +1000 (Sun, 08 Jul 2007) $
Author: Nick Coghlan <[EMAIL PROTECTED]>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 1-May-2007
Python-Version: 2.6, 3.0
Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007
Abstract
This PEP proposes a backwards compatible mechanism that permits
the use of explicit relative imports from executable modules within
packages. Such imports currently fail due to an awkward interaction
between PEP 328 and PEP 338.
By adding a new module level attribute, this PEP allows relative imports
to work automatically if the module is executed using the ``-m``switch.
A small amount of boilerplate in the module itself will allow the relative
imports to work when the file is executed by name.
Proposed Change
===
The major proposed change is the introduction of a new module level
attribute, ``__package__``. When it is present, relative imports will
be based on this attribute rather than the module ``__name__``
attribute.
As with the current ``__name__`` attribute, setting ``__package__`` will
be the responsibility of the PEP 302 loader used to import a module.
Loaders which use ``imp.new_module()`` to create the module object will
have the new attribute set automatically to
``__name__.rpartition('.')[0]``.
``runpy.run_module`` will also set the new attribute, basing it off the
``mod_name`` argument, rather than the ``run_name`` argument. This will
allow relative imports to work correctly from main modules executed with
the ``-m`` switch.
When the main module is specified by its filename, then the
``__package__`` attribute will be set to the empty string. To allow
relative imports when the module is executed directly, boilerplate
similar to the following would be needed before the first relative
import statement:
if __name__ == "__main__" and not __package_name__:
__package_name__ = ""
Note that this boilerplate is sufficient only if the top level package
is already accessible via ``sys.path``. Additional code that manipulates
``sys.path`` would be needed in order for direct execution to work
without the top level package already being importable.
This approach also has the same disadvantage as the use of absolute
imports of sibling modules - if the script is moved to a different
package or subpackage, the boilerplate will need to be updated
manually.
Rationale for Change
The current inability to use explicit relative imports from the main
module is the subject of at least one open SF bug report (#1510172)[1],
and has most likely been a factor in at least a few queries on
comp.lang.python (such as Alan Isaac's question in [2]).
This PEP is intended to provide a solution which permits explicit
relative imports from main modules, without incurring any significant
costs during interpreter startup or normal module import.
The section in PEP 338 on relative imports and the main module provides
further details and background on this problem.
Reference Implementation
Rev 47142 in SVN implemented an early variant of this proposal
which stored the main module's real module name in the
'__module_name__' attribute. It was reverted due to the fact
that 2.5 was already in beta by that time.
A new patch will be developed for 2.6, and forward ported to
Py3k via svnmerge.
Alternative Proposals
=
PEP 3122 proposed addressing this problem by changing the way
the main module is identified. That's a significant compatibility cost
to incur to fix something that is a pretty minor bug in the overall
scheme of things, and the PEP was rejected [3].
The advantage of the proposal in this PEP is that its only impact on
normal code is the small amount of time needed to set the extra
attribute when importing a module. Relative imports themselves should
be sped up fractionally, as the package name is stored in the module
glo
[Python-Dev] itertools addition: getitem()
I'd like to propose the following addition to itertools: A function itertools.getitem() which is basically equivalent to the following python code: _default = object() def getitem(iterable, index, default=_default): try: return list(iterable)[index] except IndexError: if default is _default: raise return default but without materializing the complete list. Negative indexes are supported too (this requires additional temporary storage for abs(index) objects). The patch is available at http://bugs.python.org/1749857 Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python for embedding?
> What is my idea, is to make a Python implementation made to be embedded > into applications.. > > Hope you have any ideas/comments! My main question: Who will implement that idea? Ideas are cheap; making them come true is a lot of work. It seems that you believe the current implementation of Python is *deliberately* slow, and that a new implementation would necessarily be faster. Please rest assured that this is not the case: people have put a considerable amount of work into making Python as fast as it is today. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
How important is it to have the default in this API? __getitem__() doesn't have a default; instead, there's a separate API get() that provides a default (and I find defaulting to None more manageable than the "_default = object()" pattern). --Guido On 7/8/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > I'd like to propose the following addition to itertools: A function > itertools.getitem() which is basically equivalent to the following > python code: > > _default = object() > > def getitem(iterable, index, default=_default): > try: >return list(iterable)[index] > except IndexError: >if default is _default: > raise >return default > > but without materializing the complete list. Negative indexes are > supported too (this requires additional temporary storage for abs(index) > objects). > > The patch is available at http://bugs.python.org/1749857 > > Servus, > Walter > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Guido van Rossum schrieb: > How important is it to have the default in this API? __getitem__() > doesn't have a default; instead, there's a separate API get() that > provides a default (and I find defaulting to None more manageable than > the "_default = object()" pattern). getattr() has a default too, while __getattr__ hasn't... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Georg Brandl <[EMAIL PROTECTED]> wrote: > Guido van Rossum schrieb: > > How important is it to have the default in this API? __getitem__() > > doesn't have a default; instead, there's a separate API get() that > > provides a default (and I find defaulting to None more manageable than > > the "_default = object()" pattern). > > getattr() has a default too, while __getattr__ hasn't... Fair enough. But I still want to hear of a practical use case for the default here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Guido van Rossum wrote: > On 7/8/07, Georg Brandl <[EMAIL PROTECTED]> wrote: >> Guido van Rossum schrieb: >>> How important is it to have the default in this API? __getitem__() >>> doesn't have a default; instead, there's a separate API get() that >>> provides a default (and I find defaulting to None more manageable than >>> the "_default = object()" pattern). Of course it isn't implemented this way in the C version. >> getattr() has a default too, while __getattr__ hasn't... > > Fair enough. > > But I still want to hear of a practical use case for the default here. In most cases foo = getitem(iterable, 0, None) if foo is not None: ... is simpler than: try: foo = getitem(iterable, 0) except IndexError: pass else: ... Here is a use case from one of my "import XML into the database" scripts: compid = getitem(root[ns.Company_company_id], 0, None) if compid: compid = int(compid) The expression root[ns.company_id] returns an iterator that produces all children of the root node that are of the element type company_id. If there is a company_id its content will be turned into an int, if not None will be used. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: [quoting Guido] > > But I still want to hear of a practical use case for the default here. > > In most cases > > foo = getitem(iterable, 0, None) > if foo is not None: >... > > is simpler than: > > try: >foo = getitem(iterable, 0) > except IndexError: >pass > else: >... > > Here is a use case from one of my "import XML into the database" scripts: > > compid = getitem(root[ns.Company_company_id], 0, None) > if compid: >compid = int(compid) > > The expression root[ns.company_id] returns an iterator that produces all > children of the root node that are of the element type company_id. If > there is a company_id its content will be turned into an int, if not > None will be used. Ahem. I hope you have a better use case for getitem() than that (regardless of the default issue). I find it clearer to write that as try: compid = root[ns.company_id].next() except StopIteration: compid = None else: compid = int(compid) While this is more lines, it doesn't require one to know about getitem() on an iterator. This is the same reason why setdefault() was a mistake -- it's too obscure to invent a compact spelling for it since the compact spelling has to be learned or looked up. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: Ahem. I hope you have a better use case for getitem() than that (regardless of the default issue). I find it clearer to write that as try: compid = root[ns.company_id].next() except StopIteration: compid = None else: compid = int(compid) While this is more lines, it doesn't require one to know about getitem() on an iterator. This is the same reason why setdefault() was a mistake -- it's too obscure to invent a compact spelling for it since the compact spelling has to be learned or looked up. Apropos of this discussion, I've occasionally wanted a faster version of the following: _nothing=object() def nth_next(seq,n,default=_nothing): ''' Return the n'th next element for seq, if it exists. If default is specified, it is return when the sequence is too short. Otherwise StopIteration is raised. ''' try: for i in xrange(n-1): seq.next() return seq.next() except StopIteration: if default is _nothing: raise return default The nice thing about this function is that it solves several problems in one: extraction of the n'th next element, testing for a minimum sequence length given a sentinel value, and just skipping n elements. It also leaves the sequence in a useful and predictable state, which is not true of the Python-version getitem code. While cute, I can't say if it is worthy of being an itertool function. Also vaguely apropos: def ilen(seq): 'Return the length of the hopefully finite sequence' n = 0 for x in seq: n += 1 return n Why? Because I find myself implementing it in virtually every project. Maybe I'm just an outlier, but many algorithms I implement need to consume iterators (for side-effects, obviously) and it is sometimes nice to know exactly how many elements were consumed. ~Kevin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Kevin Jacobs <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> wrote: > Also vaguely apropos: > > def ilen(seq): > 'Return the length of the hopefully finite sequence' > n = 0 > for x in seq: > n += 1 > return n Also known as:: sum(1 for _ in iterable) That's always been simple enough that I didn't feel a need for an ilen() function. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] proposed attribute lookup optimization
Hi,
I would like to propose an optimization (I think so, anyway) for the
way attributes are looked up. Currently, it is done like this:
return value of attribute in instance.__dict__ if present
for type in instance.__class__.__mro__:
return value of attribute in type.__dict__ if present
raise AttributeError
I propose adding to each type a C-implementation-private dictionary
of attribute-name => type-in-which-defined. Then, it will not be
necessary to traverse __mro__ on each attribute lookup for names
which are present in this lookup dictionary.
This optimization will not have any effect for attributes defined
on instance. It will, however, for type attributes, most notably
for methods. It will most likely cause a slowdown for looking up
attributes that are defined directly on self.__class__, not on any
of its bases. However, I believe it will be a benefit for all
non-extremely shallow inheritance tree. Especially if they involve
multiple inheritance.
One open question is what to do in case an attribute on a type is
set or deleted.
Python example:
class Current (type):
@staticmethod
def getattribute (self, name):
dict = object.__getattribute__(self, '__dict__')
if name in dict:
return dict[name]
mro = object.__getattribute__ (self, '__class__').__mro__
for type in mro:
dict = type.__dict__
if name in dict:
return dict[name]
raise AttributeError
def __init__(self, name, bases, dict):
super (Current, self).__init__(name, bases, dict)
self.__getattribute__ = Current.getattribute
class Optimized (type):
@staticmethod
def getattribute (self, name):
dict = object.__getattribute__(self, '__dict__')
if name in dict:
return dict[name]
#
lookup = object.__getattribute__ (self, '__class__').__lookup_cache__
if name in lookup:
return lookup[name].__dict__[name]
#
mro = object.__getattribute__ (self, '__class__').__mro__
for type in mro:
dict = object.__getattribute__(type, '__dict__')
if name in dict:
return dict[name]
raise AttributeError
#
def build_lookup_cache (self):
lookup = {}
for type in self.__mro__:
for name in type.__dict__:
if name not in lookup:
lookup[name] = type
return lookup
#
def __init__(self, name, bases, dict):
super (Optimized, self).__init__(name, bases, dict)
#
self.__lookup_cache__ = self.build_lookup_cache ()
#
self.__getattribute__ = Optimized.getattribute
class A (object):
__metaclass__ = Optimized
x = 1
class B (A):
pass
class C (B):
pass
class D (C):
pass
class E (D):
pass
t = E ()
for k in xrange (10):
t.x
Try swapping metaclass of A from Optimized to Current and measure
execution time.
Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] proposed attribute lookup optimization
At 08:23 PM 7/8/2007 +0300, Paul Pogonyshev wrote: >I would like to propose an optimization (I think so, anyway) for the >way attributes are looked up. Currently, it is done like this: > > return value of attribute in instance.__dict__ if present > for type in instance.__class__.__mro__: > return value of attribute in type.__dict__ if present > raise AttributeError Actually, it is only done like that for "classic" classes. New-style classes actually work more like this: descriptor = None for t in type(ob).__mro__: if attribute in t.__dict__: descriptor = t.__dict__[attribute] if hasattr(descriptor, '__set__'): return descriptor.__get__(ob, type(ob)) break if attribute in ob.__dict__: return ob.__dict__[attribute] if descriptor is not None: return descriptor.__get__(ob, type(ob)) if hasattr(type(ob),'__getattr__'): return ob.__getattr__(attribute) raise AttributeError >I propose adding to each type a C-implementation-private dictionary >of attribute-name => type-in-which-defined. Then, it will not be >necessary to traverse __mro__ on each attribute lookup for names >which are present in this lookup dictionary. Sounds good to me... but it's just as simple to store the descriptors directly, rather than the type that defines the descriptor. Might as well cut out the middleman. I believe that someone proposed this already, with a patch, in fact... >This optimization will not have any effect for attributes defined >on instance. It will for new-style classes, actually -- and a significant one if the inheritance hierarchy is deep and doesn't contain a default value for the attribute. > It will, however, for type attributes, most notably >for methods. Yep. It'll also speed up access to inherited slots. > It will most likely cause a slowdown for looking up >attributes that are defined directly on self.__class__, not on any >of its bases. Not if it's a direct cache of descriptors; in that case it will have no effect on lookup time. >One open question is what to do in case an attribute on a type is >set or deleted. New-style classes can handle this easily; they know their subclasses and you can't directly write to a new-style class' __dict__. So when you set or delete an attribute on a type, it's possible to walk the subclasses and update their caches accordingly. I believe Python already does this so that if you e.g. set 'sometype.__call__ = something', then all the subclasses' C-level tp_call slots get changed to match. The same approach could be used for caching on new-style classes. Again, though, this has already been proposed, and I believe there's a patch awaiting review for inclusion in 2.6 (and presumably 3.0). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Tracker Issues
On 7/7/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > Steve Holden <[EMAIL PROTECTED]> wrote: > > Tracker wrote: > > > > > > ACTIVITY SUMMARY (07/01/07 - 07/08/07) > > > > > > > > > Tracker at http://bugs.python.org/ > > > > > > To view or respond to any of the issues listed below, simply click on > > > the issue ID. Do *not* respond to this message. > > > > > > 1645 open ( +0) / 8584 closed ( +0) / 10229 total ( +0) > > > > > > Average duration of open issues: 850 days. > > > > > > Median duration of open issues: 798 days. > > > > > > Open Issues Breakdown STATUSNumber Change > > > open1645+0 > > > pending 0 +0 > > > > > This script appears to have been producing exactly the same output since > > June 9. I can't believe it's useful information. > > The bugs.python.org tracker is not yet the official tracker for Python > yet (partially due to the sf.net data dump issue). Yep. The email is not meant to be useful to python-dev yet. It's there as we work out issues and make sure stuff continues to work. The export issue should hopefully be fixed on Tuesday. We then need to change the importer since the DTD changed. Then we will verify everything works and do the transition (hopefully soon). -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 366 - Relative imports from main modules
On 7/8/07, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Josiah Carlson wrote:
> > "Brett Cannon" <[EMAIL PROTECTED]> wrote:
> >> On 7/5/07, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> >>> At 11:53 AM 7/5/2007 +0200, Guido van Rossum wrote:
> I see no big problems with this, except I wonder if in the end it
> wouldn't be better to *always* define __package_name__ instead of only
> when it's in main? And then perhaps rename it to __package__? Done
> properly it could always be used for relative imports, rather than
> parsing __module__ to find the package. Then you won't even need the
> error handler.
> >>> +1 for __package__, and putting it everywhere. Relative import
> >>> should use it first if present, falling back to use of __name__.
> >> +1 from me as well.
> >
> > This would solve some issues I'm currently having with relative imports.
> > +1
>
> I've updated the PEP to incorporate the feedback from this thread. The
> new version is below, and should show up on the website shortly.
>
> Cheers,
> Nick.
>
> PEP: 366
> Title: Main module explicit relative imports
> Version: $Revision: 56190 $
> Last-Modified: $Date: 2007-07-08 17:45:46 +1000 (Sun, 08 Jul 2007) $
> Author: Nick Coghlan <[EMAIL PROTECTED]>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 1-May-2007
> Python-Version: 2.6, 3.0
> Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007
>
>
> Abstract
>
>
> This PEP proposes a backwards compatible mechanism that permits
> the use of explicit relative imports from executable modules within
> packages. Such imports currently fail due to an awkward interaction
> between PEP 328 and PEP 338.
>
> By adding a new module level attribute, this PEP allows relative imports
> to work automatically if the module is executed using the ``-m``switch.
> A small amount of boilerplate in the module itself will allow the relative
> imports to work when the file is executed by name.
>
>
> Proposed Change
> ===
>
> The major proposed change is the introduction of a new module level
> attribute, ``__package__``. When it is present, relative imports will
> be based on this attribute rather than the module ``__name__``
> attribute.
>
> As with the current ``__name__`` attribute, setting ``__package__`` will
> be the responsibility of the PEP 302 loader used to import a module.
> Loaders which use ``imp.new_module()`` to create the module object will
> have the new attribute set automatically to
> ``__name__.rpartition('.')[0]``.
Is this really the best semantics for this? Let's say I have
A/B/__init__.py and A/B/C.py. With these semantics I would have A.B
having __package__ be 'A' and A.B.C having 'A.B'.
While I agree that the A.B.C setting is correct, is the A.B value what
is truly desired? Is an __init__ module really to be considered part
of the above package? I always viewed the __init__ module as part of
its own package. Thus I expected A.B to have __package__ set to
'A.B'.
Beyond just what I expected, the reason I bring this up is that if
__package__ had the semantics I am suggesting, it is trivial to
discover what modules are the package __init__ modules (as
``__package__ == __name__``) compared to being a submodule
(``__package__ and __package__ != __name__``). As of right now you
can only do that if you examine __file__ for __init__.py(c), but that
is highly dependent on how the module was loaded. It might be nice if
what kind of module (top-level, package, or submodule) something is
based on its metadata.
-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Tracker Issues
Brett Cannon wrote: > On 7/7/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: >> Steve Holden <[EMAIL PROTECTED]> wrote: >>> Tracker wrote: ACTIVITY SUMMARY (07/01/07 - 07/08/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, simply click on the issue ID. Do *not* respond to this message. 1645 open ( +0) / 8584 closed ( +0) / 10229 total ( +0) Average duration of open issues: 850 days. Median duration of open issues: 798 days. Open Issues Breakdown STATUSNumber Change open1645+0 pending 0 +0 >>> This script appears to have been producing exactly the same output since >>> June 9. I can't believe it's useful information. >> The bugs.python.org tracker is not yet the official tracker for Python >> yet (partially due to the sf.net data dump issue). > > Yep. The email is not meant to be useful to python-dev yet. It's > there as we work out issues and make sure stuff continues to work. > > The export issue should hopefully be fixed on Tuesday. We then need > to change the importer since the DTD changed. Then we will verify > everything works and do the transition (hopefully soon). > Yay! I'm sure the tracker change has taken more effort than most people expected, but it should be worth it in the end. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden --- Asciimercial -- Get on the web: Blog, lens and tag the Internet Many services currently offer free registration --- Thank You for Reading - ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Guido van Rossum wrote: > On 7/8/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > [quoting Guido] >> > But I still want to hear of a practical use case for the default here. >> >> In most cases >> >> foo = getitem(iterable, 0, None) >> if foo is not None: >>... >> >> is simpler than: >> >> try: >>foo = getitem(iterable, 0) >> except IndexError: >>pass >> else: >>... >> >> Here is a use case from one of my "import XML into the database" scripts: >> >> compid = getitem(root[ns.Company_company_id], 0, None) >> if compid: >>compid = int(compid) >> >> The expression root[ns.company_id] returns an iterator that produces all >> children of the root node that are of the element type company_id. If >> there is a company_id its content will be turned into an int, if not >> None will be used. > > Ahem. I hope you have a better use case for getitem() than that > (regardless of the default issue). I find it clearer to write that as > > try: > compid = root[ns.company_id].next() > except StopIteration: > compid = None > else: > compid = int(compid) > > While this is more lines, it doesn't require one to know about > getitem() on an iterator. This is the same reason why setdefault() was > a mistake -- it's too obscure to invent a compact spelling for it > since the compact spelling has to be learned or looked up. Well I have used (a Python version of) this getitem() function to implement a library that can match a CSS3 expression against an XML tree. For implementing the nth-child(), nth-last-child(), nth-of-type() and nth-last-of-type() pseudo classes (see http://www.w3.org/TR/css3-selectors/#structural-pseudos) getitem() was very useful. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
[Walter Dörwald] > I'd like to propose the following addition to itertools: A function > itertools.getitem() which is basically equivalent to the following > python code: > > _default = object() > > def getitem(iterable, index, default=_default): >try: > return list(iterable)[index] >except IndexError: > if default is _default: > raise > return default > > but without materializing the complete list. Negative indexes are > supported too (this requires additional temporary storage for abs(index) > objects). Why not use the existing islice() function? x = list(islice(iterable, i, i+1)) or default Also, as a practical matter, I think it is a bad idea to introduce __getitem__ style access to itertools because the starting point moves with each consecutive access: # access items 0, 2, 5, 9, 14, 20, ... for i in range(10): print getitem(iterable, i) Worse, this behavior changes depending on whether the iterable is re-iterable (a string would yield consecutive items while a generator would skip around as shown above). Besides being a bug factory, I think the getitem proposal would tend to steer people down the wrong road, away from more natural solutions to problems involving iterators. A basic step in learning the language is to differentiate between sequences and general iterators -- we should not conflate the two. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Raymond Hettinger wrote: > [Walter Dörwald] >> I'd like to propose the following addition to itertools: A function >> itertools.getitem() which is basically equivalent to the following >> python code: >> >> _default = object() >> >> def getitem(iterable, index, default=_default): >>try: >> return list(iterable)[index] >>except IndexError: >> if default is _default: >> raise >> return default >> >> but without materializing the complete list. Negative indexes are >> supported too (this requires additional temporary storage for abs(index) >> objects). > > Why not use the existing islice() function? > > x = list(islice(iterable, i, i+1)) or default This doesn't work, because it produces a list >>> list(islice(xrange(10), 2, 3)) or 42 [2] The following would work: x = (list(islice(iterable, i, i+1)) or [default])[0] However islice() doesn't support negative indexes, getitem() does. > Also, as a practical matter, I think it is a bad idea to introduce > __getitem__ style access to itertools because the starting point > moves with each consecutive access: > ># access items 0, 2, 5, 9, 14, 20, ... >for i in range(10): >print getitem(iterable, i) > > Worse, this behavior changes depending on whether the iterable > is re-iterable (a string would yield consecutive items while a > generator would skip around as shown above). islice() has the same "problem": >>> from itertools import * >>> iterable = iter(xrange(100)) >>> for i in range(10): ... print list(islice(iterable, i, i+1)) [0] [2] [5] [9] [14] [20] [27] [35] [44] [54] >>> iterable = xrange(100) >>> for i in range(10): ... print list(islice(iterable, i, i+1)) [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] > Besides being a bug factory, I think the getitem proposal would > tend to steer people down the wrong road, away from more > natural solutions to problems involving iterators. I don't think that (list(islice(iterable, i, i+1)) or [default])[0] is more natural than getitem(iterable, i, default) > A basic step > in learning the language is to differentiate between sequences > and general iterators -- we should not conflate the two. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/9/07, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > Also, as a practical matter, I think it is a bad idea to introduce > __getitem__ style access to itertools because the starting point > moves with each consecutive access: > > # access items 0, 2, 5, 9, 14, 20, ... > for i in range(10): > print getitem(iterable, i) > > Worse, this behavior changes depending on whether the iterable > is re-iterable (a string would yield consecutive items while a > generator would skip around as shown above). > > Besides being a bug factory, I think the getitem proposal would > tend to steer people down the wrong road, away from more > natural solutions to problems involving iterators. A basic step > in learning the language is to differentiate between sequences > and general iterators -- we should not conflate the two. But doesn't the very same argument also apply against islice(), which you just offered as an alternative? PS. If Walter is also at EuroPython, maybe you two could discuss this in person? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
