Re: [Python-Dev] PEP 366 - Relative imports from main modules
Josiah Carlson wrote: > "Brett Cannon" <[EMAIL PROTECTED]> wrote: >> On 7/5/07, Phillip J. Eby <[EMAIL PROTECTED]> wrote: >>> At 11:53 AM 7/5/2007 +0200, Guido van Rossum wrote: I see no big problems with this, except I wonder if in the end it wouldn't be better to *always* define __package_name__ instead of only when it's in main? And then perhaps rename it to __package__? Done properly it could always be used for relative imports, rather than parsing __module__ to find the package. Then you won't even need the error handler. >>> +1 for __package__, and putting it everywhere. Relative import >>> should use it first if present, falling back to use of __name__. >> +1 from me as well. > > This would solve some issues I'm currently having with relative imports. > +1 I've updated the PEP to incorporate the feedback from this thread. The new version is below, and should show up on the website shortly. Cheers, Nick. PEP: 366 Title: Main module explicit relative imports Version: $Revision: 56190 $ Last-Modified: $Date: 2007-07-08 17:45:46 +1000 (Sun, 08 Jul 2007) $ Author: Nick Coghlan <[EMAIL PROTECTED]> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 1-May-2007 Python-Version: 2.6, 3.0 Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007 Abstract This PEP proposes a backwards compatible mechanism that permits the use of explicit relative imports from executable modules within packages. Such imports currently fail due to an awkward interaction between PEP 328 and PEP 338. By adding a new module level attribute, this PEP allows relative imports to work automatically if the module is executed using the ``-m``switch. A small amount of boilerplate in the module itself will allow the relative imports to work when the file is executed by name. Proposed Change === The major proposed change is the introduction of a new module level attribute, ``__package__``. When it is present, relative imports will be based on this attribute rather than the module ``__name__`` attribute. As with the current ``__name__`` attribute, setting ``__package__`` will be the responsibility of the PEP 302 loader used to import a module. Loaders which use ``imp.new_module()`` to create the module object will have the new attribute set automatically to ``__name__.rpartition('.')[0]``. ``runpy.run_module`` will also set the new attribute, basing it off the ``mod_name`` argument, rather than the ``run_name`` argument. This will allow relative imports to work correctly from main modules executed with the ``-m`` switch. When the main module is specified by its filename, then the ``__package__`` attribute will be set to the empty string. To allow relative imports when the module is executed directly, boilerplate similar to the following would be needed before the first relative import statement: if __name__ == "__main__" and not __package_name__: __package_name__ = "" Note that this boilerplate is sufficient only if the top level package is already accessible via ``sys.path``. Additional code that manipulates ``sys.path`` would be needed in order for direct execution to work without the top level package already being importable. This approach also has the same disadvantage as the use of absolute imports of sibling modules - if the script is moved to a different package or subpackage, the boilerplate will need to be updated manually. Rationale for Change The current inability to use explicit relative imports from the main module is the subject of at least one open SF bug report (#1510172)[1], and has most likely been a factor in at least a few queries on comp.lang.python (such as Alan Isaac's question in [2]). This PEP is intended to provide a solution which permits explicit relative imports from main modules, without incurring any significant costs during interpreter startup or normal module import. The section in PEP 338 on relative imports and the main module provides further details and background on this problem. Reference Implementation Rev 47142 in SVN implemented an early variant of this proposal which stored the main module's real module name in the '__module_name__' attribute. It was reverted due to the fact that 2.5 was already in beta by that time. A new patch will be developed for 2.6, and forward ported to Py3k via svnmerge. Alternative Proposals = PEP 3122 proposed addressing this problem by changing the way the main module is identified. That's a significant compatibility cost to incur to fix something that is a pretty minor bug in the overall scheme of things, and the PEP was rejected [3]. The advantage of the proposal in this PEP is that its only impact on normal code is the small amount of time needed to set the extra attribute when importing a module. Relative imports themselves should be sped up fractionally, as the package name is stored in the module glo
[Python-Dev] itertools addition: getitem()
I'd like to propose the following addition to itertools: A function itertools.getitem() which is basically equivalent to the following python code: _default = object() def getitem(iterable, index, default=_default): try: return list(iterable)[index] except IndexError: if default is _default: raise return default but without materializing the complete list. Negative indexes are supported too (this requires additional temporary storage for abs(index) objects). The patch is available at http://bugs.python.org/1749857 Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python for embedding?
> What is my idea, is to make a Python implementation made to be embedded > into applications.. > > Hope you have any ideas/comments! My main question: Who will implement that idea? Ideas are cheap; making them come true is a lot of work. It seems that you believe the current implementation of Python is *deliberately* slow, and that a new implementation would necessarily be faster. Please rest assured that this is not the case: people have put a considerable amount of work into making Python as fast as it is today. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
How important is it to have the default in this API? __getitem__() doesn't have a default; instead, there's a separate API get() that provides a default (and I find defaulting to None more manageable than the "_default = object()" pattern). --Guido On 7/8/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > I'd like to propose the following addition to itertools: A function > itertools.getitem() which is basically equivalent to the following > python code: > > _default = object() > > def getitem(iterable, index, default=_default): > try: >return list(iterable)[index] > except IndexError: >if default is _default: > raise >return default > > but without materializing the complete list. Negative indexes are > supported too (this requires additional temporary storage for abs(index) > objects). > > The patch is available at http://bugs.python.org/1749857 > > Servus, > Walter > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Guido van Rossum schrieb: > How important is it to have the default in this API? __getitem__() > doesn't have a default; instead, there's a separate API get() that > provides a default (and I find defaulting to None more manageable than > the "_default = object()" pattern). getattr() has a default too, while __getattr__ hasn't... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Georg Brandl <[EMAIL PROTECTED]> wrote: > Guido van Rossum schrieb: > > How important is it to have the default in this API? __getitem__() > > doesn't have a default; instead, there's a separate API get() that > > provides a default (and I find defaulting to None more manageable than > > the "_default = object()" pattern). > > getattr() has a default too, while __getattr__ hasn't... Fair enough. But I still want to hear of a practical use case for the default here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Guido van Rossum wrote: > On 7/8/07, Georg Brandl <[EMAIL PROTECTED]> wrote: >> Guido van Rossum schrieb: >>> How important is it to have the default in this API? __getitem__() >>> doesn't have a default; instead, there's a separate API get() that >>> provides a default (and I find defaulting to None more manageable than >>> the "_default = object()" pattern). Of course it isn't implemented this way in the C version. >> getattr() has a default too, while __getattr__ hasn't... > > Fair enough. > > But I still want to hear of a practical use case for the default here. In most cases foo = getitem(iterable, 0, None) if foo is not None: ... is simpler than: try: foo = getitem(iterable, 0) except IndexError: pass else: ... Here is a use case from one of my "import XML into the database" scripts: compid = getitem(root[ns.Company_company_id], 0, None) if compid: compid = int(compid) The expression root[ns.company_id] returns an iterator that produces all children of the root node that are of the element type company_id. If there is a company_id its content will be turned into an int, if not None will be used. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: [quoting Guido] > > But I still want to hear of a practical use case for the default here. > > In most cases > > foo = getitem(iterable, 0, None) > if foo is not None: >... > > is simpler than: > > try: >foo = getitem(iterable, 0) > except IndexError: >pass > else: >... > > Here is a use case from one of my "import XML into the database" scripts: > > compid = getitem(root[ns.Company_company_id], 0, None) > if compid: >compid = int(compid) > > The expression root[ns.company_id] returns an iterator that produces all > children of the root node that are of the element type company_id. If > there is a company_id its content will be turned into an int, if not > None will be used. Ahem. I hope you have a better use case for getitem() than that (regardless of the default issue). I find it clearer to write that as try: compid = root[ns.company_id].next() except StopIteration: compid = None else: compid = int(compid) While this is more lines, it doesn't require one to know about getitem() on an iterator. This is the same reason why setdefault() was a mistake -- it's too obscure to invent a compact spelling for it since the compact spelling has to be learned or looked up. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: Ahem. I hope you have a better use case for getitem() than that (regardless of the default issue). I find it clearer to write that as try: compid = root[ns.company_id].next() except StopIteration: compid = None else: compid = int(compid) While this is more lines, it doesn't require one to know about getitem() on an iterator. This is the same reason why setdefault() was a mistake -- it's too obscure to invent a compact spelling for it since the compact spelling has to be learned or looked up. Apropos of this discussion, I've occasionally wanted a faster version of the following: _nothing=object() def nth_next(seq,n,default=_nothing): ''' Return the n'th next element for seq, if it exists. If default is specified, it is return when the sequence is too short. Otherwise StopIteration is raised. ''' try: for i in xrange(n-1): seq.next() return seq.next() except StopIteration: if default is _nothing: raise return default The nice thing about this function is that it solves several problems in one: extraction of the n'th next element, testing for a minimum sequence length given a sentinel value, and just skipping n elements. It also leaves the sequence in a useful and predictable state, which is not true of the Python-version getitem code. While cute, I can't say if it is worthy of being an itertool function. Also vaguely apropos: def ilen(seq): 'Return the length of the hopefully finite sequence' n = 0 for x in seq: n += 1 return n Why? Because I find myself implementing it in virtually every project. Maybe I'm just an outlier, but many algorithms I implement need to consume iterators (for side-effects, obviously) and it is sometimes nice to know exactly how many elements were consumed. ~Kevin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/8/07, Kevin Jacobs <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> wrote: > Also vaguely apropos: > > def ilen(seq): > 'Return the length of the hopefully finite sequence' > n = 0 > for x in seq: > n += 1 > return n Also known as:: sum(1 for _ in iterable) That's always been simple enough that I didn't feel a need for an ilen() function. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] proposed attribute lookup optimization
Hi, I would like to propose an optimization (I think so, anyway) for the way attributes are looked up. Currently, it is done like this: return value of attribute in instance.__dict__ if present for type in instance.__class__.__mro__: return value of attribute in type.__dict__ if present raise AttributeError I propose adding to each type a C-implementation-private dictionary of attribute-name => type-in-which-defined. Then, it will not be necessary to traverse __mro__ on each attribute lookup for names which are present in this lookup dictionary. This optimization will not have any effect for attributes defined on instance. It will, however, for type attributes, most notably for methods. It will most likely cause a slowdown for looking up attributes that are defined directly on self.__class__, not on any of its bases. However, I believe it will be a benefit for all non-extremely shallow inheritance tree. Especially if they involve multiple inheritance. One open question is what to do in case an attribute on a type is set or deleted. Python example: class Current (type): @staticmethod def getattribute (self, name): dict = object.__getattribute__(self, '__dict__') if name in dict: return dict[name] mro = object.__getattribute__ (self, '__class__').__mro__ for type in mro: dict = type.__dict__ if name in dict: return dict[name] raise AttributeError def __init__(self, name, bases, dict): super (Current, self).__init__(name, bases, dict) self.__getattribute__ = Current.getattribute class Optimized (type): @staticmethod def getattribute (self, name): dict = object.__getattribute__(self, '__dict__') if name in dict: return dict[name] # lookup = object.__getattribute__ (self, '__class__').__lookup_cache__ if name in lookup: return lookup[name].__dict__[name] # mro = object.__getattribute__ (self, '__class__').__mro__ for type in mro: dict = object.__getattribute__(type, '__dict__') if name in dict: return dict[name] raise AttributeError # def build_lookup_cache (self): lookup = {} for type in self.__mro__: for name in type.__dict__: if name not in lookup: lookup[name] = type return lookup # def __init__(self, name, bases, dict): super (Optimized, self).__init__(name, bases, dict) # self.__lookup_cache__ = self.build_lookup_cache () # self.__getattribute__ = Optimized.getattribute class A (object): __metaclass__ = Optimized x = 1 class B (A): pass class C (B): pass class D (C): pass class E (D): pass t = E () for k in xrange (10): t.x Try swapping metaclass of A from Optimized to Current and measure execution time. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] proposed attribute lookup optimization
At 08:23 PM 7/8/2007 +0300, Paul Pogonyshev wrote: >I would like to propose an optimization (I think so, anyway) for the >way attributes are looked up. Currently, it is done like this: > > return value of attribute in instance.__dict__ if present > for type in instance.__class__.__mro__: > return value of attribute in type.__dict__ if present > raise AttributeError Actually, it is only done like that for "classic" classes. New-style classes actually work more like this: descriptor = None for t in type(ob).__mro__: if attribute in t.__dict__: descriptor = t.__dict__[attribute] if hasattr(descriptor, '__set__'): return descriptor.__get__(ob, type(ob)) break if attribute in ob.__dict__: return ob.__dict__[attribute] if descriptor is not None: return descriptor.__get__(ob, type(ob)) if hasattr(type(ob),'__getattr__'): return ob.__getattr__(attribute) raise AttributeError >I propose adding to each type a C-implementation-private dictionary >of attribute-name => type-in-which-defined. Then, it will not be >necessary to traverse __mro__ on each attribute lookup for names >which are present in this lookup dictionary. Sounds good to me... but it's just as simple to store the descriptors directly, rather than the type that defines the descriptor. Might as well cut out the middleman. I believe that someone proposed this already, with a patch, in fact... >This optimization will not have any effect for attributes defined >on instance. It will for new-style classes, actually -- and a significant one if the inheritance hierarchy is deep and doesn't contain a default value for the attribute. > It will, however, for type attributes, most notably >for methods. Yep. It'll also speed up access to inherited slots. > It will most likely cause a slowdown for looking up >attributes that are defined directly on self.__class__, not on any >of its bases. Not if it's a direct cache of descriptors; in that case it will have no effect on lookup time. >One open question is what to do in case an attribute on a type is >set or deleted. New-style classes can handle this easily; they know their subclasses and you can't directly write to a new-style class' __dict__. So when you set or delete an attribute on a type, it's possible to walk the subclasses and update their caches accordingly. I believe Python already does this so that if you e.g. set 'sometype.__call__ = something', then all the subclasses' C-level tp_call slots get changed to match. The same approach could be used for caching on new-style classes. Again, though, this has already been proposed, and I believe there's a patch awaiting review for inclusion in 2.6 (and presumably 3.0). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Tracker Issues
On 7/7/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > Steve Holden <[EMAIL PROTECTED]> wrote: > > Tracker wrote: > > > > > > ACTIVITY SUMMARY (07/01/07 - 07/08/07) > > > > > > > > > Tracker at http://bugs.python.org/ > > > > > > To view or respond to any of the issues listed below, simply click on > > > the issue ID. Do *not* respond to this message. > > > > > > 1645 open ( +0) / 8584 closed ( +0) / 10229 total ( +0) > > > > > > Average duration of open issues: 850 days. > > > > > > Median duration of open issues: 798 days. > > > > > > Open Issues Breakdown STATUSNumber Change > > > open1645+0 > > > pending 0 +0 > > > > > This script appears to have been producing exactly the same output since > > June 9. I can't believe it's useful information. > > The bugs.python.org tracker is not yet the official tracker for Python > yet (partially due to the sf.net data dump issue). Yep. The email is not meant to be useful to python-dev yet. It's there as we work out issues and make sure stuff continues to work. The export issue should hopefully be fixed on Tuesday. We then need to change the importer since the DTD changed. Then we will verify everything works and do the transition (hopefully soon). -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 366 - Relative imports from main modules
On 7/8/07, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > "Brett Cannon" <[EMAIL PROTECTED]> wrote: > >> On 7/5/07, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > >>> At 11:53 AM 7/5/2007 +0200, Guido van Rossum wrote: > I see no big problems with this, except I wonder if in the end it > wouldn't be better to *always* define __package_name__ instead of only > when it's in main? And then perhaps rename it to __package__? Done > properly it could always be used for relative imports, rather than > parsing __module__ to find the package. Then you won't even need the > error handler. > >>> +1 for __package__, and putting it everywhere. Relative import > >>> should use it first if present, falling back to use of __name__. > >> +1 from me as well. > > > > This would solve some issues I'm currently having with relative imports. > > +1 > > I've updated the PEP to incorporate the feedback from this thread. The > new version is below, and should show up on the website shortly. > > Cheers, > Nick. > > PEP: 366 > Title: Main module explicit relative imports > Version: $Revision: 56190 $ > Last-Modified: $Date: 2007-07-08 17:45:46 +1000 (Sun, 08 Jul 2007) $ > Author: Nick Coghlan <[EMAIL PROTECTED]> > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 1-May-2007 > Python-Version: 2.6, 3.0 > Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007 > > > Abstract > > > This PEP proposes a backwards compatible mechanism that permits > the use of explicit relative imports from executable modules within > packages. Such imports currently fail due to an awkward interaction > between PEP 328 and PEP 338. > > By adding a new module level attribute, this PEP allows relative imports > to work automatically if the module is executed using the ``-m``switch. > A small amount of boilerplate in the module itself will allow the relative > imports to work when the file is executed by name. > > > Proposed Change > === > > The major proposed change is the introduction of a new module level > attribute, ``__package__``. When it is present, relative imports will > be based on this attribute rather than the module ``__name__`` > attribute. > > As with the current ``__name__`` attribute, setting ``__package__`` will > be the responsibility of the PEP 302 loader used to import a module. > Loaders which use ``imp.new_module()`` to create the module object will > have the new attribute set automatically to > ``__name__.rpartition('.')[0]``. Is this really the best semantics for this? Let's say I have A/B/__init__.py and A/B/C.py. With these semantics I would have A.B having __package__ be 'A' and A.B.C having 'A.B'. While I agree that the A.B.C setting is correct, is the A.B value what is truly desired? Is an __init__ module really to be considered part of the above package? I always viewed the __init__ module as part of its own package. Thus I expected A.B to have __package__ set to 'A.B'. Beyond just what I expected, the reason I bring this up is that if __package__ had the semantics I am suggesting, it is trivial to discover what modules are the package __init__ modules (as ``__package__ == __name__``) compared to being a submodule (``__package__ and __package__ != __name__``). As of right now you can only do that if you examine __file__ for __init__.py(c), but that is highly dependent on how the module was loaded. It might be nice if what kind of module (top-level, package, or submodule) something is based on its metadata. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Tracker Issues
Brett Cannon wrote: > On 7/7/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: >> Steve Holden <[EMAIL PROTECTED]> wrote: >>> Tracker wrote: ACTIVITY SUMMARY (07/01/07 - 07/08/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, simply click on the issue ID. Do *not* respond to this message. 1645 open ( +0) / 8584 closed ( +0) / 10229 total ( +0) Average duration of open issues: 850 days. Median duration of open issues: 798 days. Open Issues Breakdown STATUSNumber Change open1645+0 pending 0 +0 >>> This script appears to have been producing exactly the same output since >>> June 9. I can't believe it's useful information. >> The bugs.python.org tracker is not yet the official tracker for Python >> yet (partially due to the sf.net data dump issue). > > Yep. The email is not meant to be useful to python-dev yet. It's > there as we work out issues and make sure stuff continues to work. > > The export issue should hopefully be fixed on Tuesday. We then need > to change the importer since the DTD changed. Then we will verify > everything works and do the transition (hopefully soon). > Yay! I'm sure the tracker change has taken more effort than most people expected, but it should be worth it in the end. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden --- Asciimercial -- Get on the web: Blog, lens and tag the Internet Many services currently offer free registration --- Thank You for Reading - ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Guido van Rossum wrote: > On 7/8/07, Walter Dörwald <[EMAIL PROTECTED]> wrote: > [quoting Guido] >> > But I still want to hear of a practical use case for the default here. >> >> In most cases >> >> foo = getitem(iterable, 0, None) >> if foo is not None: >>... >> >> is simpler than: >> >> try: >>foo = getitem(iterable, 0) >> except IndexError: >>pass >> else: >>... >> >> Here is a use case from one of my "import XML into the database" scripts: >> >> compid = getitem(root[ns.Company_company_id], 0, None) >> if compid: >>compid = int(compid) >> >> The expression root[ns.company_id] returns an iterator that produces all >> children of the root node that are of the element type company_id. If >> there is a company_id its content will be turned into an int, if not >> None will be used. > > Ahem. I hope you have a better use case for getitem() than that > (regardless of the default issue). I find it clearer to write that as > > try: > compid = root[ns.company_id].next() > except StopIteration: > compid = None > else: > compid = int(compid) > > While this is more lines, it doesn't require one to know about > getitem() on an iterator. This is the same reason why setdefault() was > a mistake -- it's too obscure to invent a compact spelling for it > since the compact spelling has to be learned or looked up. Well I have used (a Python version of) this getitem() function to implement a library that can match a CSS3 expression against an XML tree. For implementing the nth-child(), nth-last-child(), nth-of-type() and nth-last-of-type() pseudo classes (see http://www.w3.org/TR/css3-selectors/#structural-pseudos) getitem() was very useful. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
[Walter Dörwald] > I'd like to propose the following addition to itertools: A function > itertools.getitem() which is basically equivalent to the following > python code: > > _default = object() > > def getitem(iterable, index, default=_default): >try: > return list(iterable)[index] >except IndexError: > if default is _default: > raise > return default > > but without materializing the complete list. Negative indexes are > supported too (this requires additional temporary storage for abs(index) > objects). Why not use the existing islice() function? x = list(islice(iterable, i, i+1)) or default Also, as a practical matter, I think it is a bad idea to introduce __getitem__ style access to itertools because the starting point moves with each consecutive access: # access items 0, 2, 5, 9, 14, 20, ... for i in range(10): print getitem(iterable, i) Worse, this behavior changes depending on whether the iterable is re-iterable (a string would yield consecutive items while a generator would skip around as shown above). Besides being a bug factory, I think the getitem proposal would tend to steer people down the wrong road, away from more natural solutions to problems involving iterators. A basic step in learning the language is to differentiate between sequences and general iterators -- we should not conflate the two. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
Raymond Hettinger wrote: > [Walter Dörwald] >> I'd like to propose the following addition to itertools: A function >> itertools.getitem() which is basically equivalent to the following >> python code: >> >> _default = object() >> >> def getitem(iterable, index, default=_default): >>try: >> return list(iterable)[index] >>except IndexError: >> if default is _default: >> raise >> return default >> >> but without materializing the complete list. Negative indexes are >> supported too (this requires additional temporary storage for abs(index) >> objects). > > Why not use the existing islice() function? > > x = list(islice(iterable, i, i+1)) or default This doesn't work, because it produces a list >>> list(islice(xrange(10), 2, 3)) or 42 [2] The following would work: x = (list(islice(iterable, i, i+1)) or [default])[0] However islice() doesn't support negative indexes, getitem() does. > Also, as a practical matter, I think it is a bad idea to introduce > __getitem__ style access to itertools because the starting point > moves with each consecutive access: > ># access items 0, 2, 5, 9, 14, 20, ... >for i in range(10): >print getitem(iterable, i) > > Worse, this behavior changes depending on whether the iterable > is re-iterable (a string would yield consecutive items while a > generator would skip around as shown above). islice() has the same "problem": >>> from itertools import * >>> iterable = iter(xrange(100)) >>> for i in range(10): ... print list(islice(iterable, i, i+1)) [0] [2] [5] [9] [14] [20] [27] [35] [44] [54] >>> iterable = xrange(100) >>> for i in range(10): ... print list(islice(iterable, i, i+1)) [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] > Besides being a bug factory, I think the getitem proposal would > tend to steer people down the wrong road, away from more > natural solutions to problems involving iterators. I don't think that (list(islice(iterable, i, i+1)) or [default])[0] is more natural than getitem(iterable, i, default) > A basic step > in learning the language is to differentiate between sequences > and general iterators -- we should not conflate the two. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] itertools addition: getitem()
On 7/9/07, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > Also, as a practical matter, I think it is a bad idea to introduce > __getitem__ style access to itertools because the starting point > moves with each consecutive access: > > # access items 0, 2, 5, 9, 14, 20, ... > for i in range(10): > print getitem(iterable, i) > > Worse, this behavior changes depending on whether the iterable > is re-iterable (a string would yield consecutive items while a > generator would skip around as shown above). > > Besides being a bug factory, I think the getitem proposal would > tend to steer people down the wrong road, away from more > natural solutions to problems involving iterators. A basic step > in learning the language is to differentiate between sequences > and general iterators -- we should not conflate the two. But doesn't the very same argument also apply against islice(), which you just offered as an alternative? PS. If Walter is also at EuroPython, maybe you two could discuss this in person? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com