Re: [Python-ideas] Generator syntax hooks?
On Tue, Aug 8, 2017 at 10:06 PM, Nick Coghlan wrote: > On 8 August 2017 at 09:06, Chris Barker wrote: > > It would be nice to have an easier access to an "slice iterator" though > -- > > one of these days I may write up a proposal for that. > > An idea I've occasionally toyed with [1] is some kind of "iterview" > that wraps around an arbitrary iterable and produces lazy itertools > based results rather than immediate views or copies. > > However, my experience is also that folks are *really* accustomed to > syntactic operations on containers producing either full live views > (e.g. memoryview or numpy slices, range as a dynamically computed > container), or actual copies (builtin container types). Having them > produce consumable iterators instead then gets confusing due to the > number of operations that will implicitly consume them (including > simple "x in y" checks). > > The OP's proposal doesn't fit into that category though: rather it's > asking about the case where we have an infinite iterator (e.g. > itertools.count(0)), and want to drop items until they start meeting > some condition (i.e. itertools.dropwhile) and then terminate the > iterator as soon as another condition is no longer met (i.e. > itertools.takewhile). > I don't think that's what the OP meant. The original proposal seemed to assume that it would be somehow reasonable for the input ("integers" in the example) to be able to see and parse the condition in the generator expression ("1000 <= x < 10" in the example, with "x" somehow known to be bound to the iteration value). That's at least what I think the remark "I like mathy syntax" referred to. > Right now, getting the "terminate when false" behaviour requires the > use of takewhile: > > {itertools.takewhile(lambda x: x < 100, itertools.count(1000)} > > In these cases, the standard generator expression syntax is an > attractive nuisance because it *looks* right from a mathematical > perspective, but hides an infinite loop: > > {x for x in itertools.count(0) if 1000 <= x < 100} > > The most credible proposal to address this that I've seen is to borrow > the "while" keyword in its "if not x: break" interpretation to get: > > {x for x in itertools.count(0) if 1000 <= x while x < 100} > > which would be compiled as equivalent to: > > x = set() > for x in itertools.count(0): > if 1000 <= x: > set.add(x) > if not x < 100: > break > > (and similarly for all of the other comprehension variants) > > There aren't any technical barriers I'm aware of to implementing that, > with the main historical objection being that instead of the > comprehension level while clause mapping to a while loop directly the > way the for and if clauses map to their statement level counterparts, > it would instead map to the conditional break in the expanded > loop-and-a-half form: > > while True: > if not condition: > break > > While it's taken me a long time to come around to the idea, "Make > subtle infinite loops in mathematical code easier to avoid" *is* a > pretty compelling user-focused justification for incurring that extra > complexity at the language design level. > I haven't come around to this yet. It looks like it will make explaining comprehensions more complex, since the translation of "while X" into "if not X: break" feels less direct than the translations of "for x in xs" or "if pred(x)". (In particular, your proposal seems to require more experience with mentally translating loops and conditions into jumps -- most regulars of this forum do that for a living, but I doubt it's second nature for the OP.) -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Generator syntax hooks?
On 8 August 2017 at 09:06, Chris Barker wrote: > It would be nice to have an easier access to an "slice iterator" though -- > one of these days I may write up a proposal for that. An idea I've occasionally toyed with [1] is some kind of "iterview" that wraps around an arbitrary iterable and produces lazy itertools based results rather than immediate views or copies. However, my experience is also that folks are *really* accustomed to syntactic operations on containers producing either full live views (e.g. memoryview or numpy slices, range as a dynamically computed container), or actual copies (builtin container types). Having them produce consumable iterators instead then gets confusing due to the number of operations that will implicitly consume them (including simple "x in y" checks). The OP's proposal doesn't fit into that category though: rather it's asking about the case where we have an infinite iterator (e.g. itertools.count(0)), and want to drop items until they start meeting some condition (i.e. itertools.dropwhile) and then terminate the iterator as soon as another condition is no longer met (i.e. itertools.takewhile). Right now, getting the "terminate when false" behaviour requires the use of takewhile: {itertools.takewhile(lambda x: x < 100, itertools.count(1000)} In these cases, the standard generator expression syntax is an attractive nuisance because it *looks* right from a mathematical perspective, but hides an infinite loop: {x for x in itertools.count(0) if 1000 <= x < 100} The most credible proposal to address this that I've seen is to borrow the "while" keyword in its "if not x: break" interpretation to get: {x for x in itertools.count(0) if 1000 <= x while x < 100} which would be compiled as equivalent to: x = set() for x in itertools.count(0): if 1000 <= x: set.add(x) if not x < 100: break (and similarly for all of the other comprehension variants) There aren't any technical barriers I'm aware of to implementing that, with the main historical objection being that instead of the comprehension level while clause mapping to a while loop directly the way the for and if clauses map to their statement level counterparts, it would instead map to the conditional break in the expanded loop-and-a-half form: while True: if not condition: break While it's taken me a long time to come around to the idea, "Make subtle infinite loops in mathematical code easier to avoid" *is* a pretty compelling user-focused justification for incurring that extra complexity at the language design level. Cheers, Nick. [1] https://mail.python.org/pipermail/python-ideas/2010-April/006983.html -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Pseudo methods
On 7 August 2017 at 18:48, Victor Stinner wrote: > Ruby provides this feature. A friend who is a long term user of Rails > complained that Rails abuses this and it's a mess in practice. So I > dislike this idea. Right, Python's opinionated design guidance is to clearly distinguish between "data first" designs using methods on objects and "algorithm first" designs using functools.singledispatch (or similar mechanisms), since they place different constraints on how new implementations are added, and where you should look for more information about how an algorithm works. Part of the intent behind this guidance is to better enable local reasoning about a piece of code: from my_string_utils import has_vowels if has_vowels(input("Enter a word: ")): print("Contains vowels!") else: print("Does not contain vowels!") Here, it is clear that if we want to know more about what "has_vowels" does, or if we want to request changes to how it works, then "my_string_utils" is where we need to go next. By contrast, that's significantly less clear if our string utils module were to implicitly modify the behaviour of input() or builtin strings: import my_string_utils if input("Enter a word: ").has_vowels(): print("Contains vowels!") else: print("Does not contain vowels!") To analyse and investigate this code, we need to "just know" that: - the result of "input()" doesn't normally have a "has_vowels()" method - therefore, importing "my_string_utils" must have either replaced the input builtin or mutated the str type - therefore, "my_string_utils" is probably the place to go for more information on "has_vowels" If our import line had instead looked like "import my_string_utils, my_other_utils", we'd have to go look at both of them to figure out where the "has_vowels()" method might be coming from (and hope it wasn't happening further down as a side effect of one of the modules *they* imported). Injecting methods rather than writing functions that dispatch on the type of their first argument also creates new opportunities for naming conflicts: while "my_string_utils.has_vowels" and "your_string_utils.has_vowels" can happily coexist in the same program without conflicts, there's only one "input" builtin, and only one "str" builtin. Can this level of explicitness be an obstacle at times? Yes, it can, especially for testing and interactive use, which is why Python offers features like wildcard imports, runtime support for monkeypatching of user-defined types, and runtime support for dynamically replacing builtins and module globals. However, the concerns around the difficulties of complexity management in the face of implicit action at a distance remain valid, so those features all fall into the category of "supported, but not encouraged, except in specific circumstances". Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Generator syntax hooks?
> Soni L. writes: > Steven d'Aprano writes: > > range(1000, 100) > > (x for x in range(1000, 100)) # waste of time and effort > Actually, those have different semantics! That's not real important. As Stefan Behnel points out, it's simple (and efficient) to get iterator semantics by using iter(). The big issue here is that Python is not the kind of declarative language where (x for x in int if 1_000 ≤ x ≤ 1_000_000)[1] is natural to write, let alone easy to implement efficiently. Aside from the problem of (x for x in float if 1_000 ≤ x ≤ 1_000_000) (where the answer is "just don't do that"), I can't think of any unbounded collections in Python that aren't iterables, except some types. That makes Steven's criticism pretty compelling. If you need to design a collection's __iter__ specially to allow it to decide whether the subset that satisfies some condition is exhausted, why not just subclass some appropriate existing collection with a more appropriate __iter__? Footnotes: [1] See what I did there? ;-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Generator syntax hooks?
Soni L. schrieb am 08.08.2017 um 01:56: > On 2017-08-07 08:35 PM, Steven D'Aprano wrote: >> Hi Soni, and welcome! >> >> On Mon, Aug 07, 2017 at 04:30:05PM -0300, Soni L. wrote: >> >>> What if, (x for x in integers if 1000 <= x < 100), was syntax sugar >>> for (x for x in range(1000, 100))? >> If you want the integers from 1000 to 100, use: >> >> range(1000, 100) >> >> Don't waste your time slowing down the code with an unnecessary and >> pointless wrapper that does nothing but pass every value on unchanged: >> >> (x for x in range(1000, 100)) # waste of time and effort > > Actually, those have different semantics! > x = range(1, 10) list(x) > [1, 2, 3, 4, 5, 6, 7, 8, 9] list(x) > [1, 2, 3, 4, 5, 6, 7, 8, 9] > x = (x for x in range(1, 10)) list(x) > [1, 2, 3, 4, 5, 6, 7, 8, 9] list(x) > [] In that case, use iter(range(1000, 100)). range() creates an iterable, which is iterable more than once. iter(range()) creates an iterator from that iterable, which has the semantics that you apparently wanted. Stefan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/