[Python-ideas] Re: Incremental step on road to improving situation around iterable strings
On Mar 2, 2020, at 15:13, Steven D'Aprano wrote:
>
> On Sun, Feb 23, 2020 at 01:46:53PM -0500, Richard Damon wrote:
>
>> I would agree with this. In my mind, fundamentally a 'string' is a
>> sequence of characters, not strings,
>
> If people are going to seriously propose this Character type, I think
> they need to be more concrete about the proposal and not just hand-wave
> it as "strings are sequences of characters".
I actually wrote a half-proposal on this several years back (plus a
proof-of-concept implementation that adds the chr type, but doesn’t change str
to use it or interact with it), but decided the backward compatibility problems
were too big to go forward. I can dig it up if anyone’s interested, but I can
summarize it and answer your questions from memory.
The type is called chr, it represents a single Unicode code unit, and it can be
constructed from a chr, a length-1 str, or an int. It has an __int__ method but
not an __index__. The repr is chr("A"); the str is just A. It is not
Iterable—this is the whole point, after all, that recursing on iter(element)
hits a base case.
Note that a chr can not represent an extended grapheme cluster; that would have
to be represented as a str, or as some new type that’s also a Sequence[chr].
But since Python strings don’t act like sequences of EGCs today, that’s not a
new problem. It could be used to hold code points (so bytes could also become a
sequence of chr instead of int), but that seemed too confusing (if chr(196)
could be either a UTF-8 lead byte or the single character 'Ä' depending on how
you got it… that feels like reopening the door to the same problems Python 3
eliminated).
It has some of the same methods as str, like lower, but not those that are
container-ish like find. (What about encode? translate? I can’t remember.)
Meanwhile, all the container-ish methods on str (including __contains__) can
now take a chr, but can also still take a str (in which case they still do
substring rather than containment tests). This is a little weird, but that
weirdness is already in str today (x in y does not imply any(a==x for a in y)
today; it would become true if x is chr but still not when x is str), and
convenient enough that you wouldn’t want to get rid of it.
It would be really nice to be able to construct a str from any Iterable[chr],
but of course that can’t work. So, how do you go back to a str once you’ve
converted into a different Iterable of chr? You need a new alternate
constructor classmethod like str.fromchars, I think.
IIRC, you can add chr to each other and to str, getting a str. You can also
multiply them, getting a str (even chr("A")*1 is a str, not a chr). Again, it’s
a little weird for non-sequence types to have sequence-like concat/repeat but I
don’t think it looks confusing in actual examples, and again, it’s convenient
enough that I think it’s worth it.
I considered adding and subtracting chr+int (which does the same as
chr(ord(self)+other) which makes it easier to translate a lot of C code), but
without char%int that feels incomplete, while with char%int it feels confusing
(it would be completely different from str%, and also, having % be arithmetic
but * be repeat on the same type just seems very wrong).
> Presumably you would want `mystring[0]` to return a char, not a str, but
> there are plenty of other unspecified details.
>
> - Should `mystring[0:1]`return a char or a length 1 str?
A str. That’s how all sequences work—slicing a sequence (even a len=1 slice)
never returns an element; it always returns a sequence of the same type (or,
for some third party types, a view that duck types as the same type).
> - Presumably "Z" remains a length-1 str for backward compatibility,
> so how do you create a char directly?
chr("Z") or, if you really want to, "Z"[0] would also work with two fewer
keystrokes.
> - Does `chr(n)` continue to return a str?
No, because it’s the constructor of the new type. This is a backward
compatibility problem, of course. Which could be solved by naming the type
something else, like char, and making chr still return a str, but IIRC, this
backward compatibility problem is subsumed in the larger one below, so there’s
no point fixing just this one. (Also, any new name you come up with probably
already appears in lots of existing code, which would add confusion; reusing
chr doesn’t have that problem.)
> - Is the char type a subclass of str?
No. It doesn’t support much of the str API, including all of the ABCs, so that
would badly break substitutabilty. Also, it would defeat the purpose of having
a separate type.
> - Do we support mixed concatenation between str and char?
Yes. See above.
> - If so, does concatenating the empty string to a char give a char
> or a length-1 string?
str+chr, chr+str, and chr+chr all return str, always. Just like chr*int returns
str even when the int is 1.
> - Are chars indexable?
No.
> - Do they support len()?
N
[Python-ideas] Re: None should raise a new exception, NoneError
Anyone can define their own singleton easily enough. None is unique because
it's the only built-in general-purpose singleton, and it's too late to
change that.
I have to agree with Andrew here: in a brand new language it might be a
good idea to have a unique exception to be raised for (most) operations not
defined on None. But retrofitting this into Python would be too awkward,
and it's not worth breaking backward compatibility.
(Not even in Python 4.0. :-)
On Mon, Mar 2, 2020 at 6:29 PM Christopher Barker
wrote:
> Not sure I like this idea at all, honestly, but:
>
> Maybe one of the issues we have is that None is used for multiple things
> in Python. It’s always some version of “nothing”, but what it specifically
> means depends on context.
>
> For instance, it pretty much always means “use default for keyword” when
> used as:
>
> def fun(x, y=None):
> If y is None:
> y = []
> ...
>
> But it can mean other, somewhat subtler things as well, like totally not
> defined (as opposed to default)
>
> So if it is useful to have a “raising not defined” that had a more
> specific meaning, maybe a new Singleton is in order:
>
> NotDefined, maybe?
>
> Then we could give it specific behavior without breaking any code that
> already uses None.
>
> Though I’m not the least bit convinced that the current None behavior is
> too limiting anyway.
>
> -CHB
>
>
>
>
>
> On Mon, Mar 2, 2020 at 1:46 PM Soni L. wrote:
>
>>
>>
>> On 2020-03-02 4:03 p.m., Andrew Barnert wrote:
>>
>> On Mar 2, 2020, at 09:26, Soni L.
>> wrote:
>>
>>
>> On 2020-03-02 2:04 p.m., Andrew Barnert wrote:
>>
>> On Mar 2, 2020, at 08:40, Soni L.
>> wrote:
>>
>> > > All operations on None should raise a NoneError,
>>
>>
>> So every function in every type implemented in C or in Python, whether
>> part of Python or third-party, that has code like this:
>>
>>
>> if not isisntance(arg, numbers.Integral):
>>
>> raise TypeError(f"can only spam integers, not '{arg!r}'")
>>
>>
>> … has to change to test if arg is None and raise a different error.
>> Otherwise, you’re not going to catch the error from + in your example if
>> g.foo(h) is None.
>>
>>
>> None can have __radd__ or whatnot as well, no? (read: please don't
>> directly raise TypeError, [redacted].)
>>
>>
>> That will help some types, but not all, with +.
>>
>> But, more importantly, that only helps with (reversible) operators. It
>> does nothing for int(x) raising TypeError when x is not
>> string/byteslike/number/something with __int__. Or when it doesn’t meet the
>> (different) requirements for the first or the second argument of
>> int.from_bytes. And so on for a bunch of other methods and class methods.
>> And that’s just one type; the same is true for lots of other types, and
>> plain old functions—hundreds just in the builtins, zillions in third-party
>> code.
>>
>> The fact that you have a partial solution for a tiny subset of the
>> problem doesn’t help. If you want to make all operations that raise
>> TypeError on None instead raise NoneError, you need to come up with a way
>> to do that, and I don’t see any way that doesn’t involve rewriting zillions
>> of lines of code both in Python itself and in third-party libraries and
>> applications.
>>
>> > which should be a TypeError for backwards compatibility.
>>
>>
>> But the most common errors caused by not checking for None are
>> AttributeError. If you turn these into a subclass of TypeError, that won’t
>> be backward compatible. Others are ValueError, and that won’t be backward
>> compatible either.
>>
>>
>> Hm. So, my original idea was to have a NoneError and MI (Multiple
>> Inheritance) it with TypeError, ValueError, LookupError, AttributeError,
>> etc. Then I checked "None[1]" and found that it raised TypeError, so I
>> thought all of None's unimplemented semantics provided TypeError, and
>> didn't realize attribute lookup was different. Oh well .-.
>>
>>
>> This is a pretty serious problem with the proposal. Do you have an answer
>> beyond “oh well”, or does that mean you’re giving up on the idea, or that
>> you think we should go ahead with the idea even though we know it can’t
>> work, or what?
>>
>> Where does None do ValueError, tho? I haven't seen that one.
>>
>>
>> The most recent ones I’ve seen came from NumPy and TensorFlow, both of
>> which raise ValueError from some methods if you have an array of type
>> object and have any None values, with a message like "ValueError: None
>> values not supported." I’m sure there are others; this is just the first
>> one that occurred to me.
>>
>>
>> When I was thinking about this I was thinking of the direct operations
>> like attribute, indexing, etc, so what if we don't bother with TypeError
>> and ValueError that aren't related to direct operations on None?
>>
>> e.g.:
>>
>> adding a None with something, or having None added to something, are
>> direct operations on None -> raise.
>> int(None) is *NOT* a direct operation on None -> doesn't raise. (but may
>
[Python-ideas] Re: None should raise a new exception, NoneError
Not sure I like this idea at all, honestly, but:
Maybe one of the issues we have is that None is used for multiple things in
Python. It’s always some version of “nothing”, but what it specifically
means depends on context.
For instance, it pretty much always means “use default for keyword” when
used as:
def fun(x, y=None):
If y is None:
y = []
...
But it can mean other, somewhat subtler things as well, like totally not
defined (as opposed to default)
So if it is useful to have a “raising not defined” that had a more specific
meaning, maybe a new Singleton is in order:
NotDefined, maybe?
Then we could give it specific behavior without breaking any code that
already uses None.
Though I’m not the least bit convinced that the current None behavior is
too limiting anyway.
-CHB
On Mon, Mar 2, 2020 at 1:46 PM Soni L. wrote:
>
>
> On 2020-03-02 4:03 p.m., Andrew Barnert wrote:
>
> On Mar 2, 2020, at 09:26, Soni L.
> wrote:
>
>
> On 2020-03-02 2:04 p.m., Andrew Barnert wrote:
>
> On Mar 2, 2020, at 08:40, Soni L.
> wrote:
>
> > > All operations on None should raise a NoneError,
>
>
> So every function in every type implemented in C or in Python, whether
> part of Python or third-party, that has code like this:
>
>
> if not isisntance(arg, numbers.Integral):
>
> raise TypeError(f"can only spam integers, not '{arg!r}'")
>
>
> … has to change to test if arg is None and raise a different error.
> Otherwise, you’re not going to catch the error from + in your example if
> g.foo(h) is None.
>
>
> None can have __radd__ or whatnot as well, no? (read: please don't
> directly raise TypeError, [redacted].)
>
>
> That will help some types, but not all, with +.
>
> But, more importantly, that only helps with (reversible) operators. It
> does nothing for int(x) raising TypeError when x is not
> string/byteslike/number/something with __int__. Or when it doesn’t meet the
> (different) requirements for the first or the second argument of
> int.from_bytes. And so on for a bunch of other methods and class methods.
> And that’s just one type; the same is true for lots of other types, and
> plain old functions—hundreds just in the builtins, zillions in third-party
> code.
>
> The fact that you have a partial solution for a tiny subset of the problem
> doesn’t help. If you want to make all operations that raise TypeError on
> None instead raise NoneError, you need to come up with a way to do that,
> and I don’t see any way that doesn’t involve rewriting zillions of lines of
> code both in Python itself and in third-party libraries and applications.
>
> > which should be a TypeError for backwards compatibility.
>
>
> But the most common errors caused by not checking for None are
> AttributeError. If you turn these into a subclass of TypeError, that won’t
> be backward compatible. Others are ValueError, and that won’t be backward
> compatible either.
>
>
> Hm. So, my original idea was to have a NoneError and MI (Multiple
> Inheritance) it with TypeError, ValueError, LookupError, AttributeError,
> etc. Then I checked "None[1]" and found that it raised TypeError, so I
> thought all of None's unimplemented semantics provided TypeError, and
> didn't realize attribute lookup was different. Oh well .-.
>
>
> This is a pretty serious problem with the proposal. Do you have an answer
> beyond “oh well”, or does that mean you’re giving up on the idea, or that
> you think we should go ahead with the idea even though we know it can’t
> work, or what?
>
> Where does None do ValueError, tho? I haven't seen that one.
>
>
> The most recent ones I’ve seen came from NumPy and TensorFlow, both of
> which raise ValueError from some methods if you have an array of type
> object and have any None values, with a message like "ValueError: None
> values not supported." I’m sure there are others; this is just the first
> one that occurred to me.
>
>
> When I was thinking about this I was thinking of the direct operations
> like attribute, indexing, etc, so what if we don't bother with TypeError
> and ValueError that aren't related to direct operations on None?
>
> e.g.:
>
> adding a None with something, or having None added to something, are
> direct operations on None -> raise.
> int(None) is *NOT* a direct operation on None -> doesn't raise. (but may
> still raise NoneError if we decide to do so through __int__. but honestly I
> kinda feel like __int__ is a mistake/wart and might make a separate post
> about this if there's interest.)
> indexing a None is a direct operation on None -> raise.
> indexing *with* a None is *NOT* a direct operation on None - doesn't
> raise. (this also solves some of the problems you listed below)
> etc.
>
> This seems more in-line with Python in general, and now I regret saying
> that "All operations on None should raise a NoneError" because I didn't
> realize how broad that actually was.
>
> So to refine it down a bit: "All direct operations on None, such as
> attribute access, index
[Python-ideas] Re: Incremental step on road to improving situation around iterable strings
On Mon, Feb 24, 2020 at 01:58:49PM -0800, Bruce Leban wrote: > Actually quite a bit. I write a lot of code that manipulates words to > create puzzles. In every one of those I use strings as iterables. For what > it's worth, in one of these programs, the problem I encountered was that > strings were not sufficiently performant to solve a very complex problem. I > modeled the strings as integers, and built the subset of needed string-like > operations on them -- including iteration. I am interested to hear what kind of problem this was, and what strings lacked to allow you to solve them. When you say they weren't sufficiently performant, do you mean they lacked functionality, or they had the functionality but are too slow? -- Steven ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GHQHPJHWR4EEOSEO6IDEFQSBOHYLLNSU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Incremental step on road to improving situation around iterable strings
On Sun, Feb 23, 2020 at 10:17:03PM -, Alex Hall wrote: > Steven D'Aprano wrote: > > Conceptually, we should be able to reason that every object that > > supports indexing should be iterable, without adding a special case > > exception "...except for str". > > Strings already have an exception in this area. Usually `x in y` means > `any(x == elem for elem in y)`. I don't think that there is anything specific to the `in` operator that demands that it is implemented that way. It is certainly a reasonable default implementation (I think Ruby offers it as method on a base class) but the `in` operator conceptually provides a containment test which might be far more general than the above: - number in interval - location in region - subset in set - subtree in tree - offence in criminal_record - monster in room just off the top of my head. In the case of strings, if the argument is a length-1 string, then your implementation works. If it isn't, a generalisation of it works: rather than iterate over substrings of length 1, iterate over substrings of the same length as the argument. So conceptually we have the string `in` operator being equivalent to: any(arg == sub for sub in thestring) except that iteration is not necessarily in length-1 chunks. (Note that for efficiency reasons, string containment testing may not actually be implemented in that way.) > Another somewhat related example: we usually accept that basically > every object can be treated as a boolean, even more so of it has a > `__len__`. But numpy and pandas break this 'rule' by raising an > exception if you try to treat an array as a boolean Yes, they do, and I think they are wrong to have done so. But they had their reasons, and its hard to know whether any other alternative would have been better. -- Steven ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/S6437OXHM26WFCLQVUEWA47A34GSG3G6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Incremental step on road to improving situation around iterable strings
On Tue, Mar 3, 2020 at 10:13 AM Steven D'Aprano wrote: > > On Sun, Feb 23, 2020 at 01:46:53PM -0500, Richard Damon wrote: > > > I would agree with this. In my mind, fundamentally a 'string' is a > > sequence of characters, not strings, > > If people are going to seriously propose this Character type, I think > they need to be more concrete about the proposal and not just hand-wave > it as "strings are sequences of characters". > > Presumably you would want `mystring[0]` to return a char, not a str, but > there are plenty of other unspecified details. > > - Should `mystring[0:1]`return a char or a length 1 str? I'm not seriously proposing it, and I am in fact against the proposal quite strongly, but ISTM the only sane way to do things is to mirror the Py3 bytes object. Just as mybytes[0] returns an int, not a bytes, this should return a char. And that can then be the pattern for anything else that's similar. > - Presumably "Z" remains a length-1 str for backward compatibility, > so how do you create a char directly? There would probably need to be an alternative literal form. In C, "Z" is a string, and 'Z' is a char; in Python, a more logical way to do it would probably be a prefix like c"Z" - or perhaps just "Z"[0] and have done with it. > - Does `chr(n)` continue to return a str? Logically it should return a char, and in fact would probably want to be the type, just as str/int/float etc are. > - Is the char type a subclass of str? That way lies madness. I suggest not. > - Do we support mixed concatenation between str and char? For the sake of backward compatibility, probably yes. But that's a weak opinion and could easily be swayed. > - If so, does concatenating the empty string to a char give a char > or a length-1 string? A length 1 string (or, per above, TypeError). > - Are chars indexable? > > - Do they support len()? No. A character is a single entity, just as an integer is. (NOTE: This discussion has been talking about "characters", but I think logically they have to be single Unicode codepoints. Thus the "length" of a character is not a meaningful quantity.) > If char is not a subclass of string, that's going to break code that > expects that `all(isinstance(c, str) for c in obj)` to be true when > `obj` happens to be a string. Backward compatibility WOULD be broken by this proposal (which is part of why I'm so against it). This is one of those eggs that has to be broken to make this omelette. > If char is a subclass, that means we can no longer deny that strings are > sequences of strings, since chars are strings. It also means that it > will break code that expects strings to be iterable, And that's why I say this way lies madness. > I don't have a good intuition for how much code will break or simply > stop working correctly if we changed string iteration to yield a new > char type instead of length-1 strings. > > Nor do I have a good intuition for whether this will *actually* help > much code. It seems to me that there's a good chance that this could end > up simply shifting isinstance tests for str in some contexts to > isinstance tests for char in different contexts. Agreed. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HJT6APLRZW4CSW4O6NLBRVOBLBQD6YUY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Incremental step on road to improving situation around iterable strings
On Sun, Feb 23, 2020 at 01:46:53PM -0500, Richard Damon wrote: > I would agree with this. In my mind, fundamentally a 'string' is a > sequence of characters, not strings, If people are going to seriously propose this Character type, I think they need to be more concrete about the proposal and not just hand-wave it as "strings are sequences of characters". Presumably you would want `mystring[0]` to return a char, not a str, but there are plenty of other unspecified details. - Should `mystring[0:1]`return a char or a length 1 str? - Presumably "Z" remains a length-1 str for backward compatibility, so how do you create a char directly? - Does `chr(n)` continue to return a str? - Is the char type a subclass of str? - Do we support mixed concatenation between str and char? - If so, does concatenating the empty string to a char give a char or a length-1 string? - Are chars indexable? - Do they support len()? If char is not a subclass of string, that's going to break code that expects that `all(isinstance(c, str) for c in obj)` to be true when `obj` happens to be a string. If char is a subclass, that means we can no longer deny that strings are sequences of strings, since chars are strings. It also means that it will break code that expects strings to be iterable, I don't have a good intuition for how much code will break or simply stop working correctly if we changed string iteration to yield a new char type instead of length-1 strings. Nor do I have a good intuition for whether this will *actually* help much code. It seems to me that there's a good chance that this could end up simply shifting isinstance tests for str in some contexts to isinstance tests for char in different contexts. Without a detailed proposal, I don't think we can judge how plausible this change might be. > so as you iterate over a string, > you shouldn't get another string, but a single character type (which > Python currently doesn't have). It would be totally shocking if someone > suggested that iterating a list or a tuple should return lists or tuples > of 1 element, so why do strings to this? Would it be so shocking though? If lists are *linked lists* of nodes, instead of arrays, then: - iterating over linked lists of nodes gives you nodes; - and a single node is still a list; which is not terribly different from the situation with strings. But in any case, lists are not limited to only containing other lists. That's not the case for strings. You couldn't get a dict or None contained in a string. Strings can only contain substrings, which include length-1 strings. -- Steven ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/3KYVSKMLXCL2AU4YUXDAYHZBX3X4H4YP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On 2020-03-02 4:03 p.m., Andrew Barnert wrote:
On Mar 2, 2020, at 09:26, Soni L. wrote:
On 2020-03-02 2:04 p.m., Andrew Barnert wrote:
On Mar 2, 2020, at 08:40, Soni L. wrote:
> > All operations on None should raise a NoneError,
So every function in every type implemented in C or in Python,
whether part of Python or third-party, that has code like this:
if not isisntance(arg, numbers.Integral):
raise TypeError(f"can only spam integers, not '{arg!r}'")
… has to change to test if arg is None and raise a different error.
Otherwise, you’re not going to catch the error from + in your
example if g.foo(h) is None.
None can have __radd__ or whatnot as well, no? (read: please don't
directly raise TypeError, [redacted].)
That will help some types, but not all, with +.
But, more importantly, that only helps with (reversible) operators. It
does nothing for int(x) raising TypeError when x is not
string/byteslike/number/something with __int__. Or when it doesn’t
meet the (different) requirements for the first or the second argument
of int.from_bytes. And so on for a bunch of other methods and class
methods. And that’s just one type; the same is true for lots of other
types, and plain old functions—hundreds just in the builtins, zillions
in third-party code.
The fact that you have a partial solution for a tiny subset of the
problem doesn’t help. If you want to make all operations that raise
TypeError on None instead raise NoneError, you need to come up with a
way to do that, and I don’t see any way that doesn’t involve rewriting
zillions of lines of code both in Python itself and in third-party
libraries and applications.
> which should be a TypeError for backwards compatibility.
But the most common errors caused by not checking for None are
AttributeError. If you turn these into a subclass of TypeError, that
won’t be backward compatible. Others are ValueError, and that won’t
be backward compatible either.
Hm. So, my original idea was to have a NoneError and MI (Multiple
Inheritance) it with TypeError, ValueError, LookupError,
AttributeError, etc. Then I checked "None[1]" and found that it
raised TypeError, so I thought all of None's unimplemented semantics
provided TypeError, and didn't realize attribute lookup was
different. Oh well .-.
This is a pretty serious problem with the proposal. Do you have an
answer beyond “oh well”, or does that mean you’re giving up on the
idea, or that you think we should go ahead with the idea even though
we know it can’t work, or what?
Where does None do ValueError, tho? I haven't seen that one.
The most recent ones I’ve seen came from NumPy and TensorFlow, both of
which raise ValueError from some methods if you have an array of type
object and have any None values, with a message like "ValueError: None
values not supported." I’m sure there are others; this is just the
first one that occurred to me.
When I was thinking about this I was thinking of the direct operations
like attribute, indexing, etc, so what if we don't bother with TypeError
and ValueError that aren't related to direct operations on None?
e.g.:
adding a None with something, or having None added to something, are
direct operations on None -> raise.
int(None) is *NOT* a direct operation on None -> doesn't raise. (but may
still raise NoneError if we decide to do so through __int__. but
honestly I kinda feel like __int__ is a mistake/wart and might make a
separate post about this if there's interest.)
indexing a None is a direct operation on None -> raise.
indexing *with* a None is *NOT* a direct operation on None - doesn't
raise. (this also solves some of the problems you listed below)
etc.
This seems more in-line with Python in general, and now I regret saying
that "All operations on None should raise a NoneError" because I didn't
realize how broad that actually was.
So to refine it down a bit: "All direct operations on None, such as
attribute access, indexing and mathematical operations, but not integer
conversion, should raise a NoneError". This includes weird ones like
"await None" and "yield from None" and perhaps even "with None". Does
this sound better?
[... hm, after writing this I realized I said "operations *on* None",
not "operations *with* None". it seems weird to consider "int(None)" or
"{}[None]" an operation *on* None. while they're indeed operations
*with* None, I wouldn't claim that e.g. "[].append({})" is an operation
*on* a dict just because there's a dict, so why should I do so with
None? I'd honestly say this one is on you for misunderstanding me :v]
> we can then look into merging the proposals of None-aware operators and Exception-aware
operators such that the Exception-aware operators fallback to
NoneError if no exception type is provided.
How are you going to provide an exception type for most of the
None-aware operators? For example, in a?[b][c]?[d], where do the
exception types go? W
[Python-ideas] Re: More descriptive error message than "too many values to unpack"
There was a proposal (by me) some time ago to add some structured
information to some of the Exceptions. See
https://www.python.org/dev/peps/pep-0473/, but it finally got rejected due
to being too broad. I'd be happy to revive (parts of) the proposal if
anyone is interested.
I managed though to create a PR adding a name attribute to NameError, see
https://github.com/python/cpython/pull/6271 and
https://bugs.python.org/issue37797.
On Mon, Mar 2, 2020 at 6:01 PM Alex Hall wrote:
>
>
> On Mon, Mar 2, 2020 at 12:47 AM Christopher Barker
> wrote:
>
>> That being said, more information is better than less, so maybe an
>> unpacking error should show the *value* of what was being unpacked:
>>
>> >>> a, b = 1, 2, 3
>> Traceback (most recent call last):
>> File "", line 1, in
>> ValueError: too many values to unpack (expected 2)
>>
>> Which, in fact, is what iPython already does:
>>
>> In [5]: a,b = 1,2,3
>>
>>
>> ---
>> ValueErrorTraceback (most recent call
>> last)
>> in
>> > 1 a,b = 1,2,3
>>
>> ValueError: too many values to unpack (expected 2)
>>
>
> The only extra thing IPython is doing here is showing the source line,
> which it can do because it saves the input in linecache. The standard shell
> never saves its input so it never shows in tracebacks. I do think that's an
> issue, but if you ran these examples in files you'd get the same amount of
> information either way.
>
> (cause it's showing the line of code, not the run time values)
>>
>
> There are many tools which enhance tracebacks with local variables. Even
> the standard traceback module can do it. But of course improving the
> default experience with just the right amount of information would be
> ideal.
>
>
>> So if possible, it would be great if error messages did generally show
>> the value(s) of the objects involved, if possible.
>>
>> Then that would be:
>>
>> ValueError: too many values to unpack (expected 2, got : 'this')
>>
>> Given that it's a ValueError, it seem the *value* should be known, and
>> generally relevant.
>>
>> And this is already done for some ValueErrors:
>>
>> In [3]: i = int('45f')
>>
>>
>> ---
>> ValueErrorTraceback (most recent call
>> last)
>> in
>> > 1 i = int('45f')
>>
>> ValueError: invalid literal for int() with base 10: '45f'
>>
>> Maybe it should be for ALL Value Errors?
>>
>
> In general Python error messages don't include the relevant values or much
> information about them, although I often wish they would. For example when
> I get a KeyError I wish I could see which keys are present, unless there's
> too many for it to be practical. I'm speculating, but I think there are two
> main reasons for this:
>
> 1. To avoid executing arbitrary __repr__ methods.
> 2. To avoid the performance cost of creating a message for an exception
> that may be caught and thrown away.
>
> Neither of these really apply to int('45f').
>
> Are there any other major reasons for the general lack of information? It
> feels like an intentional decision beyond implementation difficulty. I
> imagine we can work around both reasons:
>
> 1. There's a lot of information that can be extracted and displayed
> without running any user defined code.
> 2. Objects can be saved (such as the dict that raised KeyError) while
> deferring the message rendering until it's requested.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SLIFGO4FSMAM4AMZZX3B4Y3YQCNZACPE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Sebastian Kreft
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/4OYYVFNFOWY6ONW5YQAPF7R7QDS55YSL/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: More descriptive error message than "too many values to unpack"
On Mon, Mar 2, 2020 at 12:47 AM Christopher Barker
wrote:
> That being said, more information is better than less, so maybe an
> unpacking error should show the *value* of what was being unpacked:
>
> >>> a, b = 1, 2, 3
> Traceback (most recent call last):
> File "", line 1, in
> ValueError: too many values to unpack (expected 2)
>
> Which, in fact, is what iPython already does:
>
> In [5]: a,b = 1,2,3
>
> ---
> ValueErrorTraceback (most recent call last)
> in
> > 1 a,b = 1,2,3
>
> ValueError: too many values to unpack (expected 2)
>
The only extra thing IPython is doing here is showing the source line,
which it can do because it saves the input in linecache. The standard shell
never saves its input so it never shows in tracebacks. I do think that's an
issue, but if you ran these examples in files you'd get the same amount of
information either way.
(cause it's showing the line of code, not the run time values)
>
There are many tools which enhance tracebacks with local variables. Even
the standard traceback module can do it. But of course improving the
default experience with just the right amount of information would be
ideal.
> So if possible, it would be great if error messages did generally show the
> value(s) of the objects involved, if possible.
>
> Then that would be:
>
> ValueError: too many values to unpack (expected 2, got : 'this')
>
> Given that it's a ValueError, it seem the *value* should be known, and
> generally relevant.
>
> And this is already done for some ValueErrors:
>
> In [3]: i = int('45f')
>
> ---
> ValueErrorTraceback (most recent call last)
> in
> > 1 i = int('45f')
>
> ValueError: invalid literal for int() with base 10: '45f'
>
> Maybe it should be for ALL Value Errors?
>
In general Python error messages don't include the relevant values or much
information about them, although I often wish they would. For example when
I get a KeyError I wish I could see which keys are present, unless there's
too many for it to be practical. I'm speculating, but I think there are two
main reasons for this:
1. To avoid executing arbitrary __repr__ methods.
2. To avoid the performance cost of creating a message for an exception
that may be caught and thrown away.
Neither of these really apply to int('45f').
Are there any other major reasons for the general lack of information? It
feels like an intentional decision beyond implementation difficulty. I
imagine we can work around both reasons:
1. There's a lot of information that can be extracted and displayed without
running any user defined code.
2. Objects can be saved (such as the dict that raised KeyError) while
deferring the message rendering until it's requested.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/SLIFGO4FSMAM4AMZZX3B4Y3YQCNZACPE/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On Mar 2, 2020, at 09:26, Soni L. wrote:
>
>> On 2020-03-02 2:04 p.m., Andrew Barnert wrote:
>> On Mar 2, 2020, at 08:40, Soni L. wrote:
>> > > All operations on None should raise a NoneError,
>>
>> So every function in every type implemented in C or in Python, whether part
>> of Python or third-party, that has code like this:
>>
>> if not isisntance(arg, numbers.Integral):
>> raise TypeError(f"can only spam integers, not '{arg!r}'")
>>
>> … has to change to test if arg is None and raise a different error.
>> Otherwise, you’re not going to catch the error from + in your example if
>> g.foo(h) is None.
>
> None can have __radd__ or whatnot as well, no? (read: please don't directly
> raise TypeError, [redacted].)
That will help some types, but not all, with +.
But, more importantly, that only helps with (reversible) operators. It does
nothing for int(x) raising TypeError when x is not
string/byteslike/number/something with __int__. Or when it doesn’t meet the
(different) requirements for the first or the second argument of
int.from_bytes. And so on for a bunch of other methods and class methods. And
that’s just one type; the same is true for lots of other types, and plain old
functions—hundreds just in the builtins, zillions in third-party code.
The fact that you have a partial solution for a tiny subset of the problem
doesn’t help. If you want to make all operations that raise TypeError on None
instead raise NoneError, you need to come up with a way to do that, and I don’t
see any way that doesn’t involve rewriting zillions of lines of code both in
Python itself and in third-party libraries and applications.
>> > which should be a TypeError for backwards compatibility.
>>
>> But the most common errors caused by not checking for None are
>> AttributeError. If you turn these into a subclass of TypeError, that won’t
>> be backward compatible. Others are ValueError, and that won’t be backward
>> compatible either.
>
> Hm. So, my original idea was to have a NoneError and MI (Multiple
> Inheritance) it with TypeError, ValueError, LookupError, AttributeError, etc.
> Then I checked "None[1]" and found that it raised TypeError, so I thought all
> of None's unimplemented semantics provided TypeError, and didn't realize
> attribute lookup was different. Oh well .-.
This is a pretty serious problem with the proposal. Do you have an answer
beyond “oh well”, or does that mean you’re giving up on the idea, or that you
think we should go ahead with the idea even though we know it can’t work, or
what?
> Where does None do ValueError, tho? I haven't seen that one.
The most recent ones I’ve seen came from NumPy and TensorFlow, both of which
raise ValueError from some methods if you have an array of type object and have
any None values, with a message like "ValueError: None values not supported."
I’m sure there are others; this is just the first one that occurred to me.
>> > we can then look into merging the proposals of None-aware operators and
>> > Exception-aware operators such that the Exception-aware operators fallback
>> > to NoneError if no exception type is provided.
>>
>> How are you going to provide an exception type for most of the None-aware
>> operators? For example, in a?[b][c]?[d], where do the exception types go?
>> Without solving that, how do you merge the two proposals?
>>
>> > we should also look into making "except" default to using NoneError
>> > instead of BaseException, with a future flag.
>>
>> Why? That sounds like a terrible idea. Most uses of bare except are bad
>> code, but that doesn’t mean spuriously breaking all of that code and forcing
>> people to deal with a problem that may never have come up in their
>> production code is a good idea. And meanwhile, the good uses of bare
>> except—quick&dirty code at the REPL, exception handlers that access the
>> current exception through other means, etc.—would all break. And what code
>> would benefit?
>
> I don't know if these are good ideas, that's why I used the expression "look
> into".
OK, but surely you must have some idea for why it might be useful or you
wouldn’t have suggested looking into it? So what is that idea?
> fwiw, assuming we had exception-aware operators, I believe a?[b][c]?[d]
> wouldn't be possible, but I know a?[b]?[c]?[d] would become a[b][c][d]?:None
> (or a[b][c][d]?NoneError:None to be explicit.)
Even in that case it’s still not the same thing. For example, if any of b, c,
or d is None, the none-aware operators will still raise, but your
exception-aware version will not.
So, the fact that exception-aware operators could replace some but not most
uses of none-aware operators, and would be inaccurate even when they can be
used, doesn’t seem very promising.
All in all, this whole NoneError thing seems like it could be a useful design
for a brand-new Python-like language, but I can’t see how it can be retrofitted
usefully into Python._
[Python-ideas] Re: None should raise a new exception, NoneError
This feels a lot like a really verbose way of having nulls-safe operators. On Mon, Mar 2, 2020, 10:40 AM Soni L. wrote: > All operations on None should raise a NoneError, which should be a > TypeError for backwards compatibility. > > try: >x = a[b][c][d][e][f] + g.foo(h) > except NoneError: >x = None > > we can then look into merging the proposals of None-aware operators and > Exception-aware operators such that the Exception-aware operators > fallback to NoneError if no exception type is provided. > > we should also look into making "except" default to using NoneError > instead of BaseException, with a future flag. > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/FJZFLUGIGR5DWEYYUZK5LV3ULYNQSGBZ/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GPOY4VNZ7TEIE3DNDM2CWWFSY22MJLNA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
For example, In JavaScript, not defined value will return 'undefined'. This language specification makes language easier to use, but also harder to debug. I think to make NoneErorr (or going to be UndefinedError or something) in languages like JavaScript is nice. Because programmers often don't know value he/she is going to use is undefined. But in Python, we have NameError. And what you want to do can be written with NameError + AttributeErorr. I don't think this proposal is worth changing language so big. Nagata Yamato ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GVFH5HQ6ODR7CPQ4HZN3X3FVECKUJZPP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On 2020-03-02 2:04 p.m., Andrew Barnert wrote:
On Mar 2, 2020, at 08:40, Soni L. wrote:
>
> All operations on None should raise a NoneError,
So every function in every type implemented in C or in Python, whether part of
Python or third-party, that has code like this:
if not isisntance(arg, numbers.Integral):
raise TypeError(f"can only spam integers, not '{arg!r}'")
… has to change to test if arg is None and raise a different error. Otherwise,
you’re not going to catch the error from + in your example if g.foo(h) is None.
None can have __radd__ or whatnot as well, no? (read: please don't
directly raise TypeError, [redacted].)
> which should be a TypeError for backwards compatibility.
But the most common errors caused by not checking for None are AttributeError.
If you turn these into a subclass of TypeError, that won’t be backward
compatible. Others are ValueError, and that won’t be backward compatible either.
Hm. So, my original idea was to have a NoneError and MI (Multiple
Inheritance) it with TypeError, ValueError, LookupError, AttributeError,
etc. Then I checked "None[1]" and found that it raised TypeError, so I
thought all of None's unimplemented semantics provided TypeError, and
didn't realize attribute lookup was different. Oh well .-.
Where does None do ValueError, tho? I haven't seen that one.
> we can then look into merging the proposals of None-aware operators and
Exception-aware operators such that the Exception-aware operators fallback to
NoneError if no exception type is provided.
How are you going to provide an exception type for most of the None-aware
operators? For example, in a?[b][c]?[d], where do the exception types go?
Without solving that, how do you merge the two proposals?
> we should also look into making "except" default to using NoneError instead
of BaseException, with a future flag.
Why? That sounds like a terrible idea. Most uses of bare except are bad code, but
that doesn’t mean spuriously breaking all of that code and forcing people to deal
with a problem that may never have come up in their production code is a good idea.
And meanwhile, the good uses of bare except—quick&dirty code at the REPL,
exception handlers that access the current exception through other means,
etc.—would all break. And what code would benefit?
I don't know if these are good ideas, that's why I used the expression
"look into".
fwiw, assuming we had exception-aware operators, I believe a?[b][c]?[d]
wouldn't be possible, but I know a?[b]?[c]?[d] would become
a[b][c][d]?:None (or a[b][c][d]?NoneError:None to be explicit.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/FVKJ54J5ZGKNSUVL7XIR6HF6KMDLF7DG/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On Mar 2, 2020, at 08:40, Soni L. wrote:
>
> All operations on None should raise a NoneError,
So every function in every type implemented in C or in Python, whether part of
Python or third-party, that has code like this:
if not isisntance(arg, numbers.Integral):
raise TypeError(f"can only spam integers, not '{arg!r}'")
… has to change to test if arg is None and raise a different error. Otherwise,
you’re not going to catch the error from + in your example if g.foo(h) is None.
> which should be a TypeError for backwards compatibility.
But the most common errors caused by not checking for None are AttributeError.
If you turn these into a subclass of TypeError, that won’t be backward
compatible. Others are ValueError, and that won’t be backward compatible either.
> we can then look into merging the proposals of None-aware operators and
> Exception-aware operators such that the Exception-aware operators fallback to
> NoneError if no exception type is provided.
How are you going to provide an exception type for most of the None-aware
operators? For example, in a?[b][c]?[d], where do the exception types go?
Without solving that, how do you merge the two proposals?
> we should also look into making "except" default to using NoneError instead
> of BaseException, with a future flag.
Why? That sounds like a terrible idea. Most uses of bare except are bad code,
but that doesn’t mean spuriously breaking all of that code and forcing people
to deal with a problem that may never have come up in their production code is
a good idea. And meanwhile, the good uses of bare except—quick&dirty code at
the REPL, exception handlers that access the current exception through other
means, etc.—would all break. And what code would benefit?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/656LDHVFVSAKZ4YDZJYA7GKSJ342UUII/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On 2020-03-02 1:48 p.m., Steven D'Aprano wrote: On Mon, Mar 02, 2020 at 01:39:39PM -0300, Soni L. wrote: > we should also look into making "except" default to using NoneError > instead of BaseException We would we want to do that? "explicit is better than implicit" and also for consistency with the hypothetical exception-aware operators. and also because it's more commonly useful than just catching BaseException. also maybe with just NoneError + new default for "except" we can squash ppl's desires for None-aware (or exception-aware) operators! ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/P4L4C4B2PZ7LU5K3PFXSBAZVXSPCPSPE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On Tue, Mar 3, 2020 at 3:40 AM Soni L. wrote: > > All operations on None should raise a NoneError, which should be a > TypeError for backwards compatibility. > > try: >x = a[b][c][d][e][f] + g.foo(h) > except NoneError: >x = None > > we can then look into merging the proposals of None-aware operators and > Exception-aware operators such that the Exception-aware operators > fallback to NoneError if no exception type is provided. > > we should also look into making "except" default to using NoneError > instead of BaseException, with a future flag. What's the advantage? ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/D7E7GCJDRML4UOUDIILKLK4O5ENDWBZU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: None should raise a new exception, NoneError
On Mon, Mar 02, 2020 at 01:39:39PM -0300, Soni L. wrote: > we should also look into making "except" default to using NoneError > instead of BaseException We would we want to do that? -- Steven ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JIMMQDBXNJIIJB6SW42ZBOZMS2D45EFH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] None should raise a new exception, NoneError
All operations on None should raise a NoneError, which should be a TypeError for backwards compatibility. try: x = a[b][c][d][e][f] + g.foo(h) except NoneError: x = None we can then look into merging the proposals of None-aware operators and Exception-aware operators such that the Exception-aware operators fallback to NoneError if no exception type is provided. we should also look into making "except" default to using NoneError instead of BaseException, with a future flag. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FJZFLUGIGR5DWEYYUZK5LV3ULYNQSGBZ/ Code of Conduct: http://python.org/psf/codeofconduct/
