Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 2:53 AM Michael Selik wrote: > On Sun, Aug 21, 2016 at 2:46 AM eryk sun wrote: > >> On Sun, Aug 21, 2016 at 6:34 AM, Michael Selik >> wrote: >> > The detection of not hashable via __hash__ set to None was necessary, >> but >> > not desirable. Better to have never defined the method/attribute in the >> > first place. Since __iter__ isn't present on ``object``, we're free to >> use >> > the better technique of not defining __iter__ rather than defining it as >> > None, NotImplemented, etc. This is superior, because we don't want >> __iter__ >> > to show up in a dir(), help(), or other tools. >> >> The point is to be able to define __getitem__ without falling back on >> the sequence iterator. >> >> I wasn't aware of the recent commit that allows anti-registration of >> __iter__. This is perfect: >> >> >>> class C: >> ... __iter__ = None >> ... def __getitem__(self, index): return 42 >> ... >>>>> iter(C()) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: 'C' object is not iterable >> >>> isinstance(C(), collections.abc.Iterable) >> False >> > > For that to make sense, Iterable should be a parent of C, or C should be a > subclass of something registered as an Iterable. Otherwise it'd be creating > a general recommendation to say ``__iter__ = None`` on every non-Iterable > class, which would be silly. > I see your point for avoiding iterability when having __getitem__, but I hope that's seen as an anti-pattern that reduces flexibility. And I should learn to stop hitting the send button halfway through my email. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 2:46 AM eryk sun wrote: > On Sun, Aug 21, 2016 at 6:34 AM, Michael Selik > wrote: > > The detection of not hashable via __hash__ set to None was necessary, but > > not desirable. Better to have never defined the method/attribute in the > > first place. Since __iter__ isn't present on ``object``, we're free to > use > > the better technique of not defining __iter__ rather than defining it as > > None, NotImplemented, etc. This is superior, because we don't want > __iter__ > > to show up in a dir(), help(), or other tools. > > The point is to be able to define __getitem__ without falling back on > the sequence iterator. > > I wasn't aware of the recent commit that allows anti-registration of > __iter__. This is perfect: > > >>> class C: > ... __iter__ = None > ... def __getitem__(self, index): return 42 > ... >>>> iter(C()) > Traceback (most recent call last): > File "", line 1, in > TypeError: 'C' object is not iterable > >>> isinstance(C(), collections.abc.Iterable) > False > For that to make sense, Iterable should be a parent of C, or C should be a subclass of something registered as an Iterable. Otherwise it'd be creating a general recommendation to say ``__iter__ = None`` on every non-Iterable class, which would be silly. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 6:34 AM, Michael Selik wrote: > The detection of not hashable via __hash__ set to None was necessary, but > not desirable. Better to have never defined the method/attribute in the > first place. Since __iter__ isn't present on ``object``, we're free to use > the better technique of not defining __iter__ rather than defining it as > None, NotImplemented, etc. This is superior, because we don't want __iter__ > to show up in a dir(), help(), or other tools. The point is to be able to define __getitem__ without falling back on the sequence iterator. I wasn't aware of the recent commit that allows anti-registration of __iter__. This is perfect: >>> class C: ... __iter__ = None ... def __getitem__(self, index): return 42 ... >>> iter(C()) Traceback (most recent call last): File "", line 1, in TypeError: 'C' object is not iterable >>> isinstance(C(), collections.abc.Iterable) False ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 1:27 AM Chris Angelico wrote: > Hmm. It would somehow need to be recognized as "not iterable". I'm not > sure how this detection is done; is it based on the presence/absence > of __iter__, or is it by calling that method and seeing what comes > back? If the latter, then sure, an __iter__ that raises would cover > that. > The detection of not hashable via __hash__ set to None was necessary, but not desirable. Better to have never defined the method/attribute in the first place. Since __iter__ isn't present on ``object``, we're free to use the better technique of not defining __iter__ rather than defining it as None, NotImplemented, etc. This is superior, because we don't want __iter__ to show up in a dir(), help(), or other tools. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On 21 August 2016 at 16:02, eryk sun wrote: > On Sun, Aug 21, 2016 at 5:27 AM, Chris Angelico wrote: >> Hmm. It would somehow need to be recognized as "not iterable". I'm not >> sure how this detection is done; is it based on the presence/absence >> of __iter__, or is it by calling that method and seeing what comes >> back? If the latter, then sure, an __iter__ that raises would cover >> that. > > PyObject_GetIter calls __iter__ (i.e. tp_iter) if it's defined. To get > a TypeError, __iter__ can return an object that's not an iterator, > i.e. an object that doesn't have a __next__ method (i.e. tp_iternext). I believe Chris's concern was that "isintance(obj, collections.abc.Iterable)" would still return True. That's actually a valid concern, but Python 3.6 generalises the previously __hash__ specific "__hash__ = None" anti-registration mechanism to other protocols, including __iter__: https://hg.python.org/cpython/rev/72b9f195569c Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 5:27 AM, Chris Angelico wrote: > Hmm. It would somehow need to be recognized as "not iterable". I'm not > sure how this detection is done; is it based on the presence/absence > of __iter__, or is it by calling that method and seeing what comes > back? If the latter, then sure, an __iter__ that raises would cover > that. PyObject_GetIter calls __iter__ (i.e. tp_iter) if it's defined. To get a TypeError, __iter__ can return an object that's not an iterator, i.e. an object that doesn't have a __next__ method (i.e. tp_iternext). For example: >>> class C: ... def __iter__(self): return self ... def __getitem__(self, index): return 42 ... >>> iter(C()) Traceback (most recent call last): File "", line 1, in TypeError: iter() returned non-iterator of type 'C' If __iter__ isn't defined but __getitem__ is defined, then PySeqIter_New is called to get a sequence iterator. >>> class D: ... def __getitem__(self, index): return 42 ... >>> it = iter(D()) >>> type(it) >>> next(it) 42 ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On 21 August 2016 at 15:22, Nick Coghlan wrote: > There may also be a case to be made for introducing an AtomicStr type > into Python's data model that works like a normal string, but > *doesn't* support indexing, slicing, or iteration, and is instead an > opaque blob of data that nevertheless supports all the other usual > string operations. (Similar to the way that types.MappingProxyType > lets you provide a read-only view of an otherwise mutable mapping, and > that collections.KeysView, ValuesView and ItemsView provide different > interfaces for a common underlying mapping) Huh, prompted by Brendan Barnwell's comment, I just realised that a discussion I was having with Graham Dumpleton at PyCon Australia about getting the wrapt module (or equivalent functionality) into Python 3.7 (not 3.6 just due to the respective timelines) is actually relevant here: given wrapt.ObjectProxy (see http://wrapt.readthedocs.io/en/latest/wrappers.html#object-proxy ) it shouldn't actually be that difficult to write an "atomic_proxy" implementation that wraps arbitrary container objects in a proxy that permits most operations, but actively *prevents* them from being treated as collections.abc.Container instances of any kind. So if folks are looking for a way to resolve the perennial problem of "How do I tell container processing algorithms to treat *this particular container* as an opaque blob?" that arise most often with strings and binary data, I'd highly recommend that as a potentially fruitful avenue to start exploring. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 3:06 PM, Brendan Barnwell wrote: > On 2016-08-20 21:10, Chris Angelico wrote: >>> >>> >I think that while the suggestion does bring some benefit, the benefit >>> >isn't enough to make up for the code churn and disruption it would >>> >cause. But I encourage the OP to go through the standard library, pick a >>> >couple of modules, and re-write them to see how they would look using >>> >this proposal. >> >> Python still has a rule that you can iterate over anything that has >> __getitem__, and it'll be called with 0, 1, 2, 3... until it raises >> IndexError. So you have two options: Remove that rule, and require >> that all iterable objects actually define __iter__; or make strings >> non-subscriptable, which means you need to do something like >> "asdf".char_at(0) instead of "asdf"[0]. > > > Isn't the rule that that __getitem__ iteration is available only if > __iter__ is not explicitly defined? So there is a third option: retain > __getitem__ but give this new modified string type an explicit __iter__ that > raises TypeError. Hmm. It would somehow need to be recognized as "not iterable". I'm not sure how this detection is done; is it based on the presence/absence of __iter__, or is it by calling that method and seeing what comes back? If the latter, then sure, an __iter__ that raises would cover that. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On 21 August 2016 at 14:10, Chris Angelico wrote: > On Sun, Aug 21, 2016 at 12:52 PM, Steven D'Aprano wrote: >> I think that while the suggestion does bring some benefit, the benefit >> isn't enough to make up for the code churn and disruption it would >> cause. But I encourage the OP to go through the standard library, pick a >> couple of modules, and re-write them to see how they would look using >> this proposal. > > Python still has a rule that you can iterate over anything that has > __getitem__, and it'll be called with 0, 1, 2, 3... until it raises > IndexError. So you have two options: Remove that rule, and require > that all iterable objects actually define __iter__; or make strings > non-subscriptable, which means you need to do something like > "asdf".char_at(0) instead of "asdf"[0]. IMO the second option is a > total non-flyer - good luck convincing anyone that THAT is an > improvement. The first one is possible, but dramatically broadens the > backward-compatibility issue. You'd have to search for any class that > defines __getitem__ and not __iter__. That's not actually true - any type that defines __getitem__ can prevent iteration just by explicitly raising TypeError from __iter__. It would be *weird* to do so, but it's entirely possible. However, the real problem with this proposal (and the reason why the switch from 8-bit str to "bytes are effectively a tuple of ints" in Python 3 was such a pain), is that there are a lot of bytes and text processing operations that *really do* operate code point by code point. Scanning a path for directory separators, scanning a CSV (or other delimited format) for delimiters, processing regular expressions, tokenising according to a grammar, analysing words in a text for character popularity, answering questions like "Is this a valid identifier?" all involve looking at each character in a sequence individually, rather than looking at the character sequence as an atomic unit. The idiomatic pattern for doing that kind of "item by item" processing in Python is iteration (whether through the Python syntax and builtins, or through the CPython C API). Now, if we were designing a language from scratch today, there's a strong case to be made that the *right* way to represent text is to have a stream-like interface (e.g. StringIO, BytesIO) around an atomic type (e.g. CodePoint, int). But we're not designing a language from scratch - we're iterating on one with a 25 year history of design, development, and use. There may also be a case to be made for introducing an AtomicStr type into Python's data model that works like a normal string, but *doesn't* support indexing, slicing, or iteration, and is instead an opaque blob of data that nevertheless supports all the other usual string operations. (Similar to the way that types.MappingProxyType lets you provide a read-only view of an otherwise mutable mapping, and that collections.KeysView, ValuesView and ItemsView provide different interfaces for a common underlying mapping) But changing the core text type itself to no longer be suitable for use in text processing tasks? Not gonna happen :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On 2016-08-20 21:10, Chris Angelico wrote: >I think that while the suggestion does bring some benefit, the benefit >isn't enough to make up for the code churn and disruption it would >cause. But I encourage the OP to go through the standard library, pick a >couple of modules, and re-write them to see how they would look using >this proposal. Python still has a rule that you can iterate over anything that has __getitem__, and it'll be called with 0, 1, 2, 3... until it raises IndexError. So you have two options: Remove that rule, and require that all iterable objects actually define __iter__; or make strings non-subscriptable, which means you need to do something like "asdf".char_at(0) instead of "asdf"[0]. Isn't the rule that that __getitem__ iteration is available only if __iter__ is not explicitly defined? So there is a third option: retain __getitem__ but give this new modified string type an explicit __iter__ that raises TypeError. That said, I'm not sure I really support the overall proposal to change the behavior of strings. I agree that it is annoying that sometimes when you try to iterate over something you accidentally end up iterating over the characters of a string, but it's been that way for quite a while and changing it would be a significant behavior change. It seems like the main practical problem might be solved by just providing a standard library function iter_or_string or whatever, that just returns a one-item iterator if its argument is a string, or the normal iterator if not. It seems that gazillions of libraries already define such a function, and the main problem is just that, because there is no standard one, many people don't realize they need it until they accidentally iterate over a string and their code goes awry. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 12:52 PM, Steven D'Aprano wrote: >> > The fixes overall will be a lot easier and obvious than introduction of >> > unicode as default string type in Python 3.0. >> >> That's a bold claim. Have you considered what's at stake if that's not true? > > Saying that these so-called "fixes" (we haven't established yet that > Python's string behaviour is a bug that need fixing) will be easier and > more obvious than the change to Unicode is not that bold a claim. Pretty > much everything is easier and more obvious than changing to Unicode. :-) > (Possibly not bringing peace to the Middle East.) And yet it's so simple. We can teach novice programmers about two's complement [1] representations of integers, and they have no trouble comprehending that the abstract concept of "integer" is different from the concrete representation in memory. We can teach intermediate programmers how hash tables work, and how to improve their performance on CPUs with 64-byte cache lines - again, there's no comprehension barrier between "mapping from key to value" and "puddle of bytes in memory that represent that mapping". But so many programmers are entrenched in the thinking that a byte IS a character. > I think that while the suggestion does bring some benefit, the benefit > isn't enough to make up for the code churn and disruption it would > cause. But I encourage the OP to go through the standard library, pick a > couple of modules, and re-write them to see how they would look using > this proposal. Python still has a rule that you can iterate over anything that has __getitem__, and it'll be called with 0, 1, 2, 3... until it raises IndexError. So you have two options: Remove that rule, and require that all iterable objects actually define __iter__; or make strings non-subscriptable, which means you need to do something like "asdf".char_at(0) instead of "asdf"[0]. IMO the second option is a total non-flyer - good luck convincing anyone that THAT is an improvement. The first one is possible, but dramatically broadens the backward-compatibility issue. You'd have to search for any class that defines __getitem__ and not __iter__. If that *does* get considered, it wouldn't be too hard to have a compatibility function, maybe in itertools. def subscript(self): i = 0 try: while "moar indexing": yield self[i] i += 1 except IndexError: pass class Demo: def __getitem__(self, item): ... __iter__ = itertools.subscript But there'd have to be the full search of "what will this break", even before getting as far as making strings non-iterable. ChrisA [1] Not "two's compliment", although I'm told that Two can say some very nice things. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 12:34:02AM +, Michael Selik wrote: > So anything that wants to loop over a string character by character would > need to construct a new object, like ``for c in list(s)``? That seems > inefficient. I suppose you might advocate for a new type, some sort of > stringview that would allow iteration over a string, but avoid allocating > so much space as a list does, but that might bring us back to where we > started. If this was ten years ago, and we were talking about backwards incompatible changes for the soon-to-be-started Python 3000, I might be more responsive to changing strings to be an atomic type (like ints, floats, etc) with a .chars() view that iterates over the characters. Or something like what Go does (I think), namely to distinguish between Chars and Strings: indexing a string gives you a Char, and Chars are not indexable and not iterable. But even then, the change probably would have required a PEP. > > The fixes overall will be a lot easier and obvious than introduction of > > unicode as default string type in Python 3.0. > > That's a bold claim. Have you considered what's at stake if that's not true? Saying that these so-called "fixes" (we haven't established yet that Python's string behaviour is a bug that need fixing) will be easier and more obvious than the change to Unicode is not that bold a claim. Pretty much everything is easier and more obvious than changing to Unicode. :-) (Possibly not bringing peace to the Middle East.) I think that while the suggestion does bring some benefit, the benefit isn't enough to make up for the code churn and disruption it would cause. But I encourage the OP to go through the standard library, pick a couple of modules, and re-write them to see how they would look using this proposal. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016 at 5:27 PM Alexander Heger wrote: > - any code that subscripts, slices, or iterates over a str will break >> > > I would try to keep indexing and slicing, but not iterating. > So anything that wants to loop over a string character by character would need to construct a new object, like ``for c in list(s)``? That seems inefficient. I suppose you might advocate for a new type, some sort of stringview that would allow iteration over a string, but avoid allocating so much space as a list does, but that might bring us back to where we started. > The fixes overall will be a lot easier and obvious than introduction of > unicode as default string type in Python 3.0. > That's a bold claim. Have you considered what's at stake if that's not true? Anyway, why don't you write a proof of concept module for a non-iterable string, throw it on PyPI, and see if people like using it? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sun, Aug 21, 2016 at 12:28 AM Alexander Heger wrote:
> Did I leave anything out?
>> How would you weigh the benefits against the problems?
>> How would you manage the upgrade path for code that's been broken?
>>
>
> FIrst one needs to add the extension string attributes like
> split()/split(''), chars(), and substring[] (Python 3.7).
>
> When indexing becomes disallowed (Python 3.10 / 4.0) attempts to iterate
> (or slice) will raise TypeError. The fixes overall will be a lot easier
> and obvious than introduction of unicode as default string type in Python
> 3.0. It could already be used/test starting with Python 3.7 using 'from
> future import __monolythic_strings__`.
>
> Is there any equivalent __future__ import with such deep semantic
implications? Most imports I can think of are mainly syntactic.
And what would it do? change the type of string literals? change the
behavior of str methods locally in this module? globally? How will this
play with 3rd party libraries?
Sounds like it will break stuff in a way that cannot be locally fixed.
~Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
>
> So you can't lose iteration without also losing subscripting.
>>>
>>
>> Python here does a lot of things implicitly. I always felt the
>> (explicit) index operator in strings in many other languages sort of is
>> syntactic sugar, in python it was taken to do literally the same things as
>> on other objects. But it does not have to be that way.
>>
>
> You can quibble with the original design choice, but unless you borrow
> Guido's time machine, there's not much point to that discussion.
>
Just to be clear, at the time it was designed, it surely was a genious idea
with its obvious simplicity.
I spend much of my time refactoring codes and interfaces from previous
"genius" ideas, as usage matures.
> Instead, let's talk about the benefits and problems that your change
> proposal would cause.
>
> Benefits:
> - no more accidentally using str as an iterable
>
> Problems:
> - any code that subscripts, slices, or iterates over a str will break
>
I would try to keep indexing and slicing, but not iterating. Though there
have been comments that may not be straightforward to implement. Not sure
if strings would need to acquire a "substring" attribute that can be
indexed and sliced.
Did I leave anything out?
> How would you weigh the benefits against the problems?
> How would you manage the upgrade path for code that's been broken?
>
FIrst one needs to add the extension string attributes like
split()/split(''), chars(), and substring[] (Python 3.7).
When indexing becomes disallowed (Python 3.10 / 4.0) attempts to iterate
(or slice) will raise TypeError. The fixes overall will be a lot easier
and obvious than introduction of unicode as default string type in Python
3.0. It could already be used/test starting with Python 3.7 using 'from
future import __monolythic_strings__`.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016 at 4:57 PM Alexander Heger wrote: > So you can't lose iteration without also losing subscripting. >> > > Python here does a lot of things implicitly. I always felt the (explicit) > index operator in strings in many other languages sort of is syntactic > sugar, in python it was taken to do literally the same things as on other > objects. But it does not have to be that way. > You can quibble with the original design choice, but unless you borrow Guido's time machine, there's not much point to that discussion. Instead, let's talk about the benefits and problems that your change proposal would cause. Benefits: - no more accidentally using str as an iterable Problems: - any code that subscripts, slices, or iterates over a str will break Did I leave anything out? How would you weigh the benefits against the problems? How would you manage the upgrade path for code that's been broken? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
> > > Yes, I am aware it will cause a lot of backward incompatibilities... > > Tell me, would you retain the ability to subscript a string to get its > characters? > > >>> "asdf"[0] > 'a' > > If not, you break a ton of code. If you do, they are automatically > iterable *by definition*. Watch: > > class IsThisIterable: > def __getitem__(self, idx): > if idx < 5: return idx*idx > raise IndexError > > >>> iti = IsThisIterable() > >>> for x in iti: print(x) > ... > 0 > 1 > 4 > 9 > 16 > > So you can't lose iteration without also losing subscripting. > Python here does a lot of things implicitly. I always felt the (explicit) index operator in strings in many other languages sort of is syntactic sugar, in python it was taken to do literally the same things as on other objects. But it does not have to be that way. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
> > I was not proposing a character type, only that strings are not iterable: > > for i in 'abc': > print(i) > > TypeError: 'str' object is not iterable > but for i in 'abc'.chars(): print(i) a b c ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
>
> Agreed. One of the handy traits of cross-platform code is that MANY
> languages let you subscript a double-quoted string to get a
> single-quoted character. Compare these blocks of code:
>
> if ("asdf"[0] == 'a')
> write("The first letter of asdf is a.\n");
>
> if ("asdf"[0] == 'a'):
> print("The first letter of asdf is a.")
>
> if ("asdf"[0] == 'a')
> console.log("The first letter of asdf is a.")
>
> if ("asdf"[0] == 'a')
> printf("The first letter of asdf is a.\n");
>
> if ("asdf"[0] == 'a')
> echo("The first letter of asdf is a.\n");
>
> Those are Pike, Python, JavaScript/ECMAScript, C/C++, and PHP,
> respectively. Two of them treat single-quoted and double-quoted
> strings identically (Python and JS). Two use double quotes for strings
> and single quotes for character (aka integer) constants (Pike and C).
> One has double quotes for interpolated and single quotes for
> non-interpolated strings (PHP). And just to mess you up completely,
> two (or three) of these define strings to be sequences of bytes (C/C++
> and PHP, plus Python 2), two as sequences of Unicode codepoints
> (Python and Pike), and one as sequences of UTF-16 code units (JS). But
> in all five, subscripting a double-quoted string yields a
> single-quoted character.
>
> I'm firmly of the opinion that this should not change. Code clarity is
> not helped by creating a brand-new "character" type and not having a
> corresponding literal for it, and the one obvious literal, given the
> amount of prior art using it, would be some form of quote character -
> probably the apostrophe. Since that's not available, I think a
> character type would be a major hurdle to get over.
I was not proposing a character type, only that strings are not iterable:
for i in 'abc':
print(i)
TypeError: 'str' object is not iterable
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
>
> It isn't so much that strings are special, it's that lists and tuples are
> special. Very few iterables can be directly converted to Numpy arrays. Try
> `np.array({1,2})` and you get `array({1, 2}, dtype=object)`, a
> 0-dimensional array.
>
there is no representation for sets as unordered data as in Numpy all is
ordered
> > But it does deal with strings as monolithic objects,
>
> Seems to me that Numpy treats strings as "I, uh, don't really know what
> you want me to do with this" objects. That kinda makes sense for Numpy,
> because, uh, what DO you want Numpy to do with strings?
>
if it was an iterable, convert to an array length-one characters
> Numpy is NOT designed to mess around with strings, and Numpy does NOT have
> as high a proportion of programmers using it for strings, so Numpy does not
> have much insight into what's good and what's useful for programmers who
> need to mess around with strings.
>
sometimes arrays of string-like objects are needed.
In summary, Numpy isn't a good example of "strings done right, through more
> experience", because they are barely "done" at all.
>
> > doing away with many of the pitfalls of strings in Python.
>
> Please start listing the pitfalls, and show how alternatives will be an
> improvement.
>
The question is about determination of iterable objects. This has been
discussed many times on this list.
def f(x):
try:
for i in x:
f(i)
except:
print(x)
f((1,2,3))
f(('house', 'car', 'cup'))
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016, at 13:26, אלעזר wrote: > It's possible (though not so pretty or obvious) to use $a for the > character "a". This isn't Lisp. If I were inventing a character literal for Python I would probably spell it c'a'. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/19/2016 2:57 PM, Guido van Rossum wrote: I don't think we should take action now. Would it make sense, as a precaution, to declare the PEP provisional for one release? Then we can develop a sense of whether the current approach causes real problems. We could also emit some kind of warning if the expression part contains an escaped quote, since that's where a potential change would cause breakage. (Or we could leave that to the linters.) If anything, I'd make it an error to have any backslashes inside the brackets of an f-string for 3.6. We could always remove this restriction at a later date. I say this because as soon as f-strings make it into the wild, we're going to have a hard time breaking code in say 3.7 by saying "well, we told you that f-strings might change". Although frankly, other than be executive fiat (which I'm okay with), I don't see us ever resolving the issue if f-strings are strings first, or if the brackets put you into "non-string mode". There are good arguments on both sides. Moving to the implementation details, I'm not sure how easy it would be to even find backslashes, though. IIRC, backslashes are replaced early, before the f-string parser really gets to process the string. It might require a new implementation of the f-string parser independent of regular strings, which I likely would not have time for before beta 1. Although since this would be a reduction in functionality, maybe it doesn't have to get done by then. I also haven't thought of how this would affect raw f-strings. In any event, I'll take a look at adding this restriction, just to get an estimate of the magnitude of work involved. The easiest thing to do might be to disallow backslashes in any part of an f-string for 3.6, although that seems to be going too far. Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016 at 4:54 PM Chris Angelico wrote: > I'm firmly of the opinion that this should not change. Code clarity is > not helped by creating a brand-new "character" type and not having a > corresponding literal for it, and the one obvious literal, given the > amount of prior art using it, would be some form of quote character - > probably the apostrophe. Since that's not available, I think a > character type would be a major hurdle to get over. It's possible (though not so pretty or obvious) to use $a for the character "a". ~Elazar ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Consider having collections.abc.Sequence
On Sat, Aug 20, 2016 at 3:54 PM Michael Selik wrote: > On Fri, Aug 19, 2016 at 8:55 AM Neil Girdhar > wrote: > >> Sure. >> >> http://bugs.python.org/issue27802 >> >> >> On Friday, August 19, 2016 at 8:36:39 AM UTC-4, Emanuel Barry wrote: >>> >>> Arek Bulski wrote: >>> >>> > Could use all(a==b for zip(seq,seq2)) >>> >>> >>> >>> Or even `all(itertools.starmap(operator.eq, zip(a, b)))` if you prefer, >>> but this isn’t about how easy or clever or obfuscated one can write that; >>> it’s about convenience. ABCs expose the lowest common denominator for >>> concrete classes of their kind, and having __eq__ makes sense for Sequence >>> (I’m surprised that it’s not already in). >>> >>> >>> >>> I think we can skip the Python-ideas thread and go straight to opening >>> an issue and submitting a patch :) Neil, care to do that? >>> >>> >>> >>> -Emanuel >>> >> > tuples and lists are both Sequences, yet are not equal to each other. > > py> [1] == (1,) > False > > As long as you treat them as an ABC.Sequences, they _should_ be equal. One can think of a static method Sequence.equals(a, b) for that purpose. Of course, that's not how it's done in dynamic languages such as Python (or Java!), so implementing the default __eq__ this way will break symmetry. ~Elazar ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016 at 10:31 PM, Michael Selik wrote:
> On Sat, Aug 20, 2016 at 3:48 AM Chris Angelico wrote:
>>
>> On Sat, Aug 20, 2016 at 4:28 PM, Alexander Heger wrote:
>> > Yes, I am aware it will cause a lot of backward incompatibilities...
>>
>> Tell me, would you retain the ability to subscript a string to get its
>> characters?
>>
>> >>> "asdf"[0]
>> 'a'
>
>
> A separate character type would solve that issue. While Alexander Heger was
> advocating for a "monolithic object," and may in fact not want subscripting,
> I think he's more frustrated by the fact that iterating over a string gives
> other strings. If instead a 1-length string were a different, non-iterable
> type, that might avoid some problems.
>
> However, special-casing a character as a different type would bring its own
> problems. Note the annoyance of iterating over bytes and getting integers.
>
> In case it's not clear, I should add that I disagree with this proposal and
> do not want any change to strings.
Agreed. One of the handy traits of cross-platform code is that MANY
languages let you subscript a double-quoted string to get a
single-quoted character. Compare these blocks of code:
if ("asdf"[0] == 'a')
write("The first letter of asdf is a.\n");
if ("asdf"[0] == 'a'):
print("The first letter of asdf is a.")
if ("asdf"[0] == 'a')
console.log("The first letter of asdf is a.")
if ("asdf"[0] == 'a')
printf("The first letter of asdf is a.\n");
if ("asdf"[0] == 'a')
echo("The first letter of asdf is a.\n");
Those are Pike, Python, JavaScript/ECMAScript, C/C++, and PHP,
respectively. Two of them treat single-quoted and double-quoted
strings identically (Python and JS). Two use double quotes for strings
and single quotes for character (aka integer) constants (Pike and C).
One has double quotes for interpolated and single quotes for
non-interpolated strings (PHP). And just to mess you up completely,
two (or three) of these define strings to be sequences of bytes (C/C++
and PHP, plus Python 2), two as sequences of Unicode codepoints
(Python and Pike), and one as sequences of UTF-16 code units (JS). But
in all five, subscripting a double-quoted string yields a
single-quoted character.
I'm firmly of the opinion that this should not change. Code clarity is
not helped by creating a brand-new "character" type and not having a
corresponding literal for it, and the one obvious literal, given the
amount of prior art using it, would be some form of quote character -
probably the apostrophe. Since that's not available, I think a
character type would be a major hurdle to get over.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Consider having collections.abc.Sequence
On Fri, Aug 19, 2016 at 8:55 AM Neil Girdhar wrote: > Sure. > > http://bugs.python.org/issue27802 > > > On Friday, August 19, 2016 at 8:36:39 AM UTC-4, Emanuel Barry wrote: >> >> Arek Bulski wrote: >> >> > Could use all(a==b for zip(seq,seq2)) >> >> >> >> Or even `all(itertools.starmap(operator.eq, zip(a, b)))` if you prefer, >> but this isn’t about how easy or clever or obfuscated one can write that; >> it’s about convenience. ABCs expose the lowest common denominator for >> concrete classes of their kind, and having __eq__ makes sense for Sequence >> (I’m surprised that it’s not already in). >> >> >> >> I think we can skip the Python-ideas thread and go straight to opening an >> issue and submitting a patch :) Neil, care to do that? >> >> >> >> -Emanuel >> > tuples and lists are both Sequences, yet are not equal to each other. py> [1] == (1,) False ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016 at 3:48 AM Chris Angelico wrote: > On Sat, Aug 20, 2016 at 4:28 PM, Alexander Heger wrote: > > Yes, I am aware it will cause a lot of backward incompatibilities... > > Tell me, would you retain the ability to subscript a string to get its > characters? > > >>> "asdf"[0] > 'a' > A separate character type would solve that issue. While Alexander Heger was advocating for a "monolithic object," and may in fact not want subscripting, I think he's more frustrated by the fact that iterating over a string gives other strings. If instead a 1-length string were a different, non-iterable type, that might avoid some problems. However, special-casing a character as a different type would bring its own problems. Note the annoyance of iterating over bytes and getting integers. In case it's not clear, I should add that I disagree with this proposal and do not want any change to strings. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 20 August 2016 at 05:02, Random832 wrote:
> On Fri, Aug 19, 2016, at 19:09, Paul Moore wrote:
>> So, to me
>>
>> f'{x.partition(' + ')[0]}'
>>
>> reads as a string concatenation. I'm not sure how you'd expect a
>> syntax highlighter to make it look like anything else, to be honest
>
> One possible syntax highlighting scheme:
Thanks for the detailed explanation and example. Yes, that may well be
a reasonable highlighting scheme.
I'd still object to reusing single quotes in the example given,
though, as it would be confusing if printed, or in email, etc. And as
a general principle, "needs syntax highlighting to be readable" is a
problem to me. So I stand by my statement that as a style rule,
f-strings should be written to work identically regardless of whether
this proposal is implemented or not.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 20 August 2016 at 04:57, C Anthony Risinger wrote: > The two string parts are string-colored and the x.partition bits would look > like > any other code in the file. It won't look like concatenation at that point. That's entirely subjective and theoretical (unless you've implemented it and reviewed the resulting look of the code). In my (equally subjective and theoretical) opinion it would still look like concatenation, and would confuse me. I made a deliberate point of saying that *to me* it looked like concatenation. YMMV - remember this tangent was started by people stating their opinions. Saying that your opinion differs doesn't invalidate their (my) view. > tl;dr, UX is weaker when the editor implies a free-form expression in every > possible way, but the writer can't use the quote they always use, and I > think strings will be common in f-string expression sections. FWIW I would instantly reject any code passed to me for review which used the same quote within an f-string as was used to delimit it, should this proposal be accepted. Also, a lot of code is read on media that doesn't do syntax highlighting - email, books, etc. A construct that needs syntax highlighting to be readable is problematic because of this. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
C Anthony Risinger writes: > The only time I personally use a different quote is when it somehow > makes the data more amenable to the task at hand. The data! The > literal data! Not the expressions I'm conveniently inlining with > the help of f-strings. You do *not* know that yet! *Nobody* does. Nobody has yet written an f-string in production code, let alone read thousands and written hundreds. Can you be sure that after you write a couple dozen f-strings you won't find that such "quote context" is carried over naturally from the way you write other strings? (Eg, because "I'm still in a string" is signaled by the highlighting of the surrounding stringish text.) I think the proposed changes to the PEP fall into the "Sometimes never is better than *right* now" category. The arguments I've seen so far are plausible but not founded in experience: it could easily go the other way, and I don't see potential for true disaster. > If I have to water it down for people to find it acceptable (such > as creating simpler variables ahead-of-time) I'd probably just keep > using .format(...). Because what I have gained with an f-string? I don't see a problem if you choose not to write f-strings. Would other people using that convention be hard for you to *read*? > Not just because it's at odds with other languages, but because > it's at odds with what the editor is telling the user (free-form > expression). There are no editors that will tell you such a thing yet. And if you trust an editor that *does* tell you that it's a free-form expression and use the same kind of quote that delimits the f-string, you won't actually create a syntax error. You're merely subject to the same kind of "problem" that you have if you don't write PEP8 conforming code. Regards, ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fix default encodings on Windows
Chris Barker writes: > Sure -- but it's entirely unnecessary, yes? If you don't change > your code, you'll get py2(bytes) strings as paths in py2, and py3 > (Unicode) strings as paths on py3. So different, yes. But wouldn't > it all work? The difference is that if you happen to have a file name on Unix that is *not* encoded in the default locale, bytes Just Works, while Something Bad happens with unicode (mixing Python 3 and Python 2 terminology for clarity). Also, in Python the C/POSIX default locale implied a codec of 'ascii' which is quite risky nowadays, so using unicode meant always being conscious of encodings. > So folks are making an active choice to change their code to get some > perceived (real?) performance benefit??? No, they're making a passive choice to not fix whut ain't broke nohow, but in Python 3 is spelled differently. It's the same order of change as "print stuff" (Python 2) to "print(stuff)" (Python 3), except that it's not as automatic. (Ie, where print is *always* a function call in Python 3, often in a Python 2 -> 3 port you're better off with str than bytes, especially before PEP 461 "% formatting for bytes".) > However, as I understand it, py3 string paths did NOT "just work" > in place of py2 paths before surrogate pairs were introduced (when > was that?) I'm not sure what you're referring to. Python 2 unicode and Python 3 str have been capable of representing (for values of "representing" that require appropriate choice of I/O codecs) the entire repertoire of Unicode since version 1.6 [sic!]. I suppose you mean PEP 383 (implemented in Python 3.1), which added a pseudo-encoding for unencodable bytes, ie, the surrogateescape error handler. This was never a major consideration in practice, however, as you could always get basically the same effect with the 'latin-1' codec. That is, the surrogateescape handler is primarily of benefit to those who are already convinced that fully conformant Unicode is the way to go. It doesn't make a difference to those who prefer bytes. > What I'm getting at is whether there is anything other than inertia > that keeps folks using bytes paths in py3 code? Maybe it wouldn't > be THAT hard to get folks to make the switch: it's EASIER to port > your code to py3 this way! It's not. First, encoding awareness is real work. If you try to DTRT, you open yourself up to UnicodeErrors anywhere in your code where there's a Python/rest-of-world boundary. If you just use bytes, you may be producing garbage, but your program doesn't stop running, and you can always argue it's either your upstream's or your downstream's fault. I *personally* have always found the work to be worthwhile, as my work always involves "real" text processing, and frequently not in pure ASCII. Second, there are a lot of low-level use cases where (1) efficiency matters and (2) all the processing actually done involves switching on byte values in the range 32-126. It makes sense to do that work on bytes, wouldn't you say? And to make the switch cases easier to read, it's common practice to form (or contort) those bytes into human words. These cases include a lot of the familiar acronyms: SMTP, HTTP, DNS, VCS, VM (as in "bytecode interpreter"), ... and the projects are familiar: Twisted, Mercurial, Bottom line: I'm with you! I think that "filenames are text" *should* be the default mode for Python programmers. But there are important use cases where it's sometimes more work to make that work than to make bytes work (on POSIX), and typically those cases also inherit largish, battle-tested code bases that assume a "bytes in, bytes through, bytes out" model. We can't deprecate "filenames as bytes" on POSIX yet, and if we want to encourage participation in projects that use that model by Windows-based programmers, we can't deprecate completely on Windows, either. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Sat, Aug 20, 2016 at 4:28 PM, Alexander Heger wrote: > Yes, I am aware it will cause a lot of backward incompatibilities... Tell me, would you retain the ability to subscript a string to get its characters? >>> "asdf"[0] 'a' If not, you break a ton of code. If you do, they are automatically iterable *by definition*. Watch: class IsThisIterable: def __getitem__(self, idx): if idx < 5: return idx*idx raise IndexError >>> iti = IsThisIterable() >>> for x in iti: print(x) ... 0 1 4 9 16 So you can't lose iteration without also losing subscripting. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] discontinue iterable strings
On Aug 20, 2016 2:27 AM, "Alexander Heger" wrote:
> The point is it does not try to disassemble it into elements as it would
do with other iterables
>
> >>> np.array([1,2,3])
> array([1, 2, 3])
> >>> np.array([1,2,3]).shape
> (3,)
It isn't so much that strings are special, it's that lists and tuples are
special. Very few iterables can be directly converted to Numpy arrays. Try
`np.array({1,2})` and you get `array({1, 2}, dtype=object)`, a
0-dimensional array.
> But it does deal with strings as monolithic objects,
Seems to me that Numpy treats strings as "I, uh, don't really know what you
want me to do with this" objects. That kinda makes sense for Numpy,
because, uh, what DO you want Numpy to do with strings?
Numpy is NOT designed to mess around with strings, and Numpy does NOT have
as high a proportion of programmers using it for strings, so Numpy does not
have much insight into what's good and what's useful for programmers who
need to mess around with strings.
In summary, Numpy isn't a good example of "strings done right, through more
experience", because they are barely "done" at all.
> doing away with many of the pitfalls of strings in Python.
Please start listing the pitfalls, and show how alternatives will be an
improvement.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
