Re: [Python-ideas] More user-friendly version for string.translate()

[Chris Barker]

On Fri, Oct 28, 2016 at 7:28 AM, Terry Reedy  wrote:

> >>> s = 'kjskljkxcvnalsfjaweirKJZknzsnlkjsvnskjszsdscccjasfdjf'
> >>> s2 = ''.join(c for c in s if c in set('abc'))
>

pretty slick -- but any hope of it being as fast as a C implemented method?

for example, with a 1000 char string:

In [59]: % timeit string.translate(table)
10 loops, best of 3: 3.62 µs per loop


In [60]: % timeit ''.join(c for c in string if c in set(letters))
1000 loops, best of 3: 1.14 ms per loop

so the translate() method is about 300 times faster in this case. (and it
used a defaultdict with a None factory, which is probably a bit slower than
a pure C implementation might be.

I've always figured that Python's rich string methods provided two things:

1) single method call to do common things

2) nice fast, pure C performance

so I think a "keep these" method would help with both of these goals.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Guido van Rossum]

Obviously the AST needs to be changed. How? I dunno. Sounds like you have
some ideas. :-)

On Mon, Oct 31, 2016 at 2:44 PM, Random832  wrote:

> On Mon, Oct 31, 2016, at 13:16, Guido van Rossum wrote:
> > For "everything to the right" it would seem we have some freedom: e.g.
> > if we have "foo.bar?.baz(bletch)" is the call included? The answer is
> > yes -- the concept we're after here is named "trailer" in the Grammar
> > file in the source code (
> > https://github.com/python/cpython/blob/master/Grammar/Grammar#L119),
> > and "primary" in the reference manual (
> > https://docs.python.org/3/reference/expressions.html#primaries). This
> > means all attribute references ("x.y"), index/slice operations
> > ("x[...]"), and calls ("x(...)").
>
> One thing that I think I touched on in an earlier iteration of this
> discussion but hasn't been revisited is: what's the AST going to look
> like?
>
> Right now, foo.bar.baz(bletch) is Call(Attribute(Attribute(Name('foo'),
> 'bar'), 'baz'), [Name('bletch')])), which is identical to
> (foo.bar).baz(bletch) or (foo.bar.baz)(bletch). These are treated,
> essentially, as postfix operators, where you can parenthesize any left
> part of the expression and leave its meaning [and its AST] unchanged.
>
> Is the AST going to be unchanged, leading to the conclusion that the
> short-circuiting in (foo?.bar).baz will "reach outside of" the
> parentheses, and relying on the fact that wanting to do that with None
> is a silly thing to do in almost all cases? Or is there going to be a
> new kind of AST that is sequential rather than recursive in how it
> represents trailer/primary expressions?
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Random832]

On Mon, Oct 31, 2016, at 13:16, Guido van Rossum wrote:
> For "everything to the right" it would seem we have some freedom: e.g.
> if we have "foo.bar?.baz(bletch)" is the call included? The answer is
> yes -- the concept we're after here is named "trailer" in the Grammar
> file in the source code (
> https://github.com/python/cpython/blob/master/Grammar/Grammar#L119),
> and "primary" in the reference manual (
> https://docs.python.org/3/reference/expressions.html#primaries). This
> means all attribute references ("x.y"), index/slice operations
> ("x[...]"), and calls ("x(...)").

One thing that I think I touched on in an earlier iteration of this
discussion but hasn't been revisited is: what's the AST going to look
like?

Right now, foo.bar.baz(bletch) is Call(Attribute(Attribute(Name('foo'),
'bar'), 'baz'), [Name('bletch')])), which is identical to
(foo.bar).baz(bletch) or (foo.bar.baz)(bletch). These are treated,
essentially, as postfix operators, where you can parenthesize any left
part of the expression and leave its meaning [and its AST] unchanged.

Is the AST going to be unchanged, leading to the conclusion that the
short-circuiting in (foo?.bar).baz will "reach outside of" the
parentheses, and relying on the fact that wanting to do that with None
is a silly thing to do in almost all cases? Or is there going to be a
new kind of AST that is sequential rather than recursive in how it
represents trailer/primary expressions?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[MRAB]

On 2016-10-31 17:16, Guido van Rossum wrote:

I think we should try to improve our intutition about these operators.
Many things that are intuitively clear still require long turgid
paragraphs in reference documentation to state the behavior
unambiguously -- but that doesn't stop us from intuitively grasping
concepts like a+b (when does b.__radd__ get called?) or @classmethod.

The main case to build intuition for is "?." -- once you get that the
"?[...]" operator works just the same. So here's my attempt:

*In a series of attribute accesses like "foo.bar.baz.bletch", putting a
`?` before a specific dot inserts a None check for the expression to the
left and skips everything to the right when the None check is true.*

We still need to clarify what we mean by "expression to the left" and
"everything to the right", but in most situations you will guess right
without thinking about it.

The expression to the left is easy -- it's determined by syntactic
operator precedence, so that if we have "x = y + foo.bar?.baz.bletch",
the expression to the left of the "?." is just "foo.bar". (But see below
-- you won't actually see such usage much.)

For "everything to the right" it would seem we have some freedom: e.g.
if we have "foo.bar?.baz(bletch)" is the call included? The answer is
yes -- the concept we're after here is named "trailer" in the Grammar
file in the source code
(https://github.com/python/cpython/blob/master/Grammar/Grammar#L119),
and "primary" in the reference manual
(https://docs.python.org/3/reference/expressions.html#primaries). This
means all attribute references ("x.y"), index/slice operations
("x[...]"), and calls ("x(...)").

Note that in almost all cases the "?." operator will be used in an
context where there is no other operator of lower precedence before or
after it -- given the above meaning, it doesn't make a lot of sense to
write "1 + x?.a" because "1 + None" is always an error (and ditto for
"x?.a + 1"). However it still makes sense to assign such an expression
to a variable or pass it as an argument to a function.

So you can ignore the preceding four paragraphs: just remember the
simplified rule (indented and in bold, depending on your email client)
and let your intuition do the rest. Maybe it can even be simplified more:

*The "?." operator splits the expression in two parts; the second part
is skipped if the first part is None.
*

Eventually this *will* become intuitive. The various constraints are all
naturally imposed by the grammar so you won't have to think about them
consciously.


Would it help if we referred to them collectively as "suffixes"?

Coincidentally, David Mertz's post includes a case where this feature 
would shorten the code.


In normal Python form his code has:

if x in stop_on or (end_if and end_if(x)):

With this feature it could be:

if x in stop_on or end_if?(x):

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Paul Moore]

On 31 October 2016 at 17:16, Guido van Rossum  wrote:
> I think we should try to improve our intutition about these operators. Many
> things that are intuitively clear still require long turgid paragraphs in
> reference documentation to state the behavior unambiguously -- but that
> doesn't stop us from intuitively grasping concepts like a+b (when does
> b.__radd__ get called?) or @classmethod.
[...]
> The "?." operator splits the expression in two parts; the second part is
> skipped if the first part is None.
>
> Eventually this *will* become intuitive. The various constraints are all
> naturally imposed by the grammar so you won't have to think about them
> consciously.

Thanks. Yes, I agree that details in a spec are never particularly
obvious, and we need an intuition of what the operator does if it's to
be successful.

Mark - I couldn't offer a specific rewording, precisely because I
found the whole thing confusing. But based on Guido's post, I'd say
that the "intuitive" explanation of the proposed operators should be
right at the top of the PEP, in the abstract - and should be repeated
as the first statement in the specification section for each operator.
The details can then follow, including all of the corner cases. But
I'd be inclined there to word the corner cases as positive statements,
rather than negative ones. Take for example, the case
"d?.year.numerator + 1" - you say

"""
Note that the error in the second example is not on the attribute
access numerator . In fact, that attribute access is never performed.
The error occurs when adding None + 1 , because the None -aware
attribute access does not short circuit + .
"""

which reads to me as presenting the misconception (that the error was
from the access to numerator) before the correct explanation, and then
explaining to the reader why they were confused if they thought that.
I'd rather see it worded something along the lines of:

"""

>>> d = date.today()
>>> d?.year.numerator + 1
2016

>>> d = None
>>> d?.year.numerator + 1

Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Note that the second example demonstrates that when ?. splits the
enclosing expression into 2 parts, operators like + have a lower
precedence, and so are not short circuited. So, we get a TypeError if
d is None, because we're trying to add None to an integer (as the
error states).
"""

There's no need to even *mention* the incorrect interpretation, it
does nothing for people who'd misinterpreted the example in the first
place, but for people who hadn't, it just suggests to them an
alternative explanation they hadn't thought of - so confusing them
where they weren't confused before.

Does this clarify what I was struggling with in the way the PEP was worded?

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Guido van Rossum]

I think we should try to improve our intutition about these operators. Many
things that are intuitively clear still require long turgid paragraphs in
reference documentation to state the behavior unambiguously -- but that
doesn't stop us from intuitively grasping concepts like a+b (when does
b.__radd__ get called?) or @classmethod.

The main case to build intuition for is "?." -- once you get that the
"?[...]" operator works just the same. So here's my attempt:

*In a series of attribute accesses like "foo.bar.baz.bletch", putting a `?`
before a specific dot inserts a None check for the expression to the left
and skips everything to the right when the None check is true.*

We still need to clarify what we mean by "expression to the left" and
"everything to the right", but in most situations you will guess right
without thinking about it.

The expression to the left is easy -- it's determined by syntactic operator
precedence, so that if we have "x = y + foo.bar?.baz.bletch", the
expression to the left of the "?." is just "foo.bar". (But see below -- you
won't actually see such usage much.)

For "everything to the right" it would seem we have some freedom: e.g. if
we have "foo.bar?.baz(bletch)" is the call included? The answer is yes --
the concept we're after here is named "trailer" in the Grammar file in the
source code (
https://github.com/python/cpython/blob/master/Grammar/Grammar#L119), and
"primary" in the reference manual (
https://docs.python.org/3/reference/expressions.html#primaries). This means
all attribute references ("x.y"), index/slice operations ("x[...]"), and
calls ("x(...)").

Note that in almost all cases the "?." operator will be used in an context
where there is no other operator of lower precedence before or after it --
given the above meaning, it doesn't make a lot of sense to write "1 + x?.a"
because "1 + None" is always an error (and ditto for "x?.a + 1"). However
it still makes sense to assign such an expression to a variable or pass it
as an argument to a function.

So you can ignore the preceding four paragraphs: just remember the
simplified rule (indented and in bold, depending on your email client) and
let your intuition do the rest. Maybe it can even be simplified more:


*The "?." operator splits the expression in two parts; the second part is
skipped if the first part is None.*

Eventually this *will* become intuitive. The various constraints are all
naturally imposed by the grammar so you won't have to think about them
consciously.

--Guido

On Mon, Oct 31, 2016 at 9:33 AM, Paul Moore  wrote:

> On 31 October 2016 at 15:51, Mark E. Haase  wrote:
> > Therefore, I have updated the PEP with the punctuation mentioned above,
> and
> > at this point the PEP can't go any farther. If the best spelling for this
> > new operator is unacceptable, then there's no getting around that. This
> PEP
> > should be rejected.
>
> While I agree that there's little point arguing over spelling here -
> if the ? spelling is unacceptable we should just reject - I'm not sure
> that's the only sticking point remaining here. I still find the
> short-circuiting behaviour of ?. (and ?[) to be pretty confusing - and
> the fact that there's a long paragraph describing the behaviour, with
> lots of examples of the form "if you think that this example works
> like so, then you're wrong, and it actually does the following",
> suggests to me that I'm not going to be the only one struggling.
> Hopefully, the problem is simply the way the behaviour is presented,
> and a reworking of the description would make it all crystal clear -
> but it feels to me that there's some inherent complexity here that's
> an actual issue with the proposal.
>
> Having said that, it appears that the proposed behaviour is the same
> as in C# (can you just come out and say "C#", rather than hinting with
> the phrase "other popular languages" - if we're stealing the behaviour
> as is from C#, let's say so, and if not, can you include examples from
> more than one language?) Assuming that's the case, then the fact that
> it's not causing confusion to C# programmers is a definite point in
> its favour.
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Mark E. Haase]

The PEP combines ideas from several different languages. For example:

* Both Dart and C# have "x ?? y" and "x?.y".
* Dart has "x ??= y" and C# does not.
* C# has short circuit semantics for "?." and Dart does not.
* PHP has "??" but does not have "?."
* Etc.

Wikipedia lists a lot of other languages[1], but I don't have enough
personal experience with any of them to cite them in the PEP. This is why I
use the phrase "other mainstream languages" multiple times.

If you think the safe navigation operator isn't presented clearly, I am
willing to improve it. Is there a particular example that you're struggling
with? The simplest explanation is that it works the way you would want it
too, e.g. in "foo?.bar.baz", we don't want semantics that could lead to
looking up "baz" as an attribute of None. Therefore, if "foo?.bar"
evaluates to None, then ".baz" is short circuited — that attribute is not
looked up.

[1] https://en.wikipedia.org/wiki/Null_coalescing_operator#SQL

On Mon, Oct 31, 2016 at 12:33 PM, Paul Moore  wrote:

> On 31 October 2016 at 15:51, Mark E. Haase  wrote:
> > Therefore, I have updated the PEP with the punctuation mentioned above,
> and
> > at this point the PEP can't go any farther. If the best spelling for this
> > new operator is unacceptable, then there's no getting around that. This
> PEP
> > should be rejected.
>
> While I agree that there's little point arguing over spelling here -
> if the ? spelling is unacceptable we should just reject - I'm not sure
> that's the only sticking point remaining here. I still find the
> short-circuiting behaviour of ?. (and ?[) to be pretty confusing - and
> the fact that there's a long paragraph describing the behaviour, with
> lots of examples of the form "if you think that this example works
> like so, then you're wrong, and it actually does the following",
> suggests to me that I'm not going to be the only one struggling.
> Hopefully, the problem is simply the way the behaviour is presented,
> and a reworking of the description would make it all crystal clear -
> but it feels to me that there's some inherent complexity here that's
> an actual issue with the proposal.
>
> Having said that, it appears that the proposed behaviour is the same
> as in C# (can you just come out and say "C#", rather than hinting with
> the phrase "other popular languages" - if we're stealing the behaviour
> as is from C#, let's say so, and if not, can you include examples from
> more than one language?) Assuming that's the case, then the fact that
> it's not causing confusion to C# programmers is a definite point in
> its favour.
>
> Paul
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Mark E. Haase]

Stephen J. Turnbull wrote:

> I gather you think you have a deadlock here.  The way to break it is
> to just do it.  Pick a syntax and do the rewriting.  My memory of some
> past instances is that many of the senior devs (especially Guido) will
> "see through the syntax" to evaluate the benefits of the proposal,
> even if they've said they don't particularly like the initially-
> proposed syntax.


I don't feel deadlocked, but I think you're right about committing to a
syntax. So I updated the PEP, summarized here:

   1. Spelling a new operator as a keyword is difficult due to backward
   compatibility. It can be done (see PEP-308 and PEP-492) but requires
   extreme care.
   2. A keyword operator is considered less ugly than punctuation, but it
   makes assignment shortcut syntax very ugly. Assume the coalesce operator is
   "foo", then the assignment shortcut would be "x foo= y". This is
   unacceptable.
   3. If eliminate the possibility of a keyword and focus on punctuation,
   we find that most people think "??" — the syntax that exists in several
   other mainstream languages — is ugly and not Pythonic.
   4. However, any other punctuation spelling will be at least as ugly and
   will not have the benefit of being familiar to programmers who have seen
   null coalescing in other languages.
   5. Therefore, the most reasonable spelling is to borrow the same
   spelling that other languages use, e.g. "??", "?.", and "?[".

I did go down the road of trying to create a new keyword, trying some
mundane ideas ("foo else bar") and some more exotic ideas ("try foo then
bar"), but I don't know if those syntaxes are even parseable, and as I
worked through a bunch of examples, I realized that all of the keywords I
was trying were very awkward in practical use, especially when combined
with other expressions.

Therefore, I have updated the PEP with the punctuation mentioned above, and
at this point the PEP can't go any farther. If the best spelling for this
new operator is unacceptable, then there's no getting around that. This PEP
should be rejected.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Non-ASCII in Python syntax? [was: Null coalescing operator]

[Steven D'Aprano]

On Mon, Oct 31, 2016 at 10:19:58AM +0900, Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
> 
>  > I see that Perl is leading the way here, supporting a large number of 
>  > Unicode symbols:
>  > 
>  > https://docs.perl6.org/language/unicode_entry.html
> 
> In what sense is that "support"?  

In the sense that Perl 6 not only allows Unicode identifiers (as Python 
has for many years) but also Unicode operators and symbols. For example, 
you can use either the Unicode character ⊂ \N{SUBSET OF} or the ASCII 
trigraph (<) for doing subset tests.


>  > I must say that it is kinda cute that Perl6 does the right thing for x².
> 
> Uh, as far as I can tell from that page, Perl has absolutely nothing
> to do with that.  You enter the Unicode code point as hex, and if the
> font supports, you get the character.

You missed the bit that Parl 6 interprets "x²" in code as the equivalent 
of x**2 (x squared).

In other words, ² behaves as a unary postfix operator that squares its 
argument. Likewise for ³, etc. You can even combine them: x³³ would be 
the same as x**33. There's more here:

https://docs.perl6.org/language/unicode_texas




-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Non-ASCII in Python syntax? [was: Null coalescing operator]

[Steven D'Aprano]

On Sun, Oct 30, 2016 at 10:16:19AM -0700, David Mertz wrote:

> My vim configuration for a year or two has looked something like this (the
> screenshot doesn't show the empty set symbol, but that's part of my conceal
> configuration: http://gnosis.cx/bin/.vim/after/syntax/python.vim).

Oh nice!

By the way, anyone looking at this in a browser may find that the 
browser defaults to treating it as Latin-1, which gives you mojibake. 
Just tell you browser to treat it as Unicode.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Non-ASCII in Python syntax? [was: Null coalescing operator]

[David Mertz]

My vim configuration for a year or two has looked something like this (the
screenshot doesn't show the empty set symbol, but that's part of my conceal
configuration: http://gnosis.cx/bin/.vim/after/syntax/python.vim).

On Sun, Oct 30, 2016 at 7:13 AM, Chris Angelico  wrote:

> On Mon, Oct 31, 2016 at 12:39 AM, Paul Moore  wrote:
> > It's certainly not difficult, in principle. I have (had, I lost it in
> > an upgrade recently...) a little AutoHotkey program that interpreted
> > Vim-style digraphs in any application that needed them. But my point
> > was that we don't want to require people to write such custom
> > utilities, just to be able to write Python code. Or is the feeling
> > that it's acceptable to require that?
>
> There's a chicken-and-egg problem. So long as most people don't have
> tools like that, a language that requires them is going to be very
> annoying - but so long as no major language uses such characters,
> there's no reason for developers to set up those kinds of tools.
>
> Possibly the best way is a gentle introduction of alternative
> syntaxes. Since Python currently has no "empty set display" syntax,
> that seems like a perfect starting point. You can always type "set()",
> but that involves an actual function call; using ∅ gives a small
> performance boost, eliminates the risk of shadowing, etc, etc. All
> minor points, but could be convenient enough. Also, if repr(set())
> returns "∅", it'll be easy for anyone to get hold of the character for
> copy/paste.
>
> As of 2016, I think it's not acceptable to *require* this, but it may
> be time to start making use of it, retaining ASCII-only digraphs and
> trigraphs, the way C has alternative spelling for braces and so on.
> Then time passes, most people will be comfortable using the characters
> themselves, and the digraphs/trigraphs can be deprecated, with new
> syntax not being given any.
>
> Pipe dream?
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/